-- Logs begin at Tue 2022-07-19 11:51:32 UTC, end at Tue 2022-07-19 13:18:54 UTC. -- Jul 19 12:16:49.651602 dv-cinder-820994-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 19 12:16:49.660661 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 19 12:16:49 2022] *** Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: nodename: dv-cinder-820994-4-iscsi Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: machine: x86_64 Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: clock source: unix Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: pcre jit disabled Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: detected number of CPU cores: 4 Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: current working directory: / Jul 19 12:16:49.662574 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: detected binary path: /usr/bin/uwsgi-core Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: your processes number limit is 47680 Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: your memory page size is 4096 bytes Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: detected max file descriptor number: 2048 Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: lock engine: pthread robust mutexes Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: thunder lock: enabled Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 19 12:16:49.663742 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 19 12:16:49.706166 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: Python main interpreter initialized at 0x564220e1e9c0 Jul 19 12:16:49.706166 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: python threads support enabled Jul 19 12:16:49.706166 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: your server socket listen backlog is limited to 100 connections Jul 19 12:16:49.706166 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: your mercy for graceful operations on workers is 90 seconds Jul 19 12:16:49.706489 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: mapped 403077 bytes (393 KB) for 2 cores Jul 19 12:16:49.706571 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: *** Operational MODE: preforking *** Jul 19 12:16:49.706846 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: *** uWSGI is running in multiple interpreter mode *** Jul 19 12:16:49.706846 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: spawned uWSGI master process (pid: 80163) Jul 19 12:16:49.707032 dv-cinder-820994-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 19 12:16:49.708557 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: spawned uWSGI worker 1 (pid: 80170, cores: 1) Jul 19 12:16:49.708557 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: spawned uWSGI worker 2 (pid: 80171, cores: 1) Jul 19 12:16:49.708557 dv-cinder-820994-4-iscsi devstack@keystone.service[80163]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 19 12:16:55.083557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 12:16:55.083557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: from cryptography.utils import int_from_bytes Jul 19 12:16:55.084352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 12:16:55.084352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: from cryptography.utils import int_from_bytes Jul 19 12:16:55.084466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 12:16:55.084466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 19 12:16:55.084466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 19 12:16:55.084466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 19 12:16:55.133162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 12:16:55.133364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 12:16:55.133447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 19 12:16:55.133562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] command line args: [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 12:16:55.133644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 19 12:16:55.133807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 19 12:16:55.133896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 12:16:55.133995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ================================================================================ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 12:16:55.134101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 19 12:16:55.134248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] admin_token = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.134328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 19 12:16:55.134491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_dir = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.134579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.134678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.134809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.134888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] config_source = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.135003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.135082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] debug = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.135189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.135275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.136591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.136591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.136823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.136823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_config_append = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_dir = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.138917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.139951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.140931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.141893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.142010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.142124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.142252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.142252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] publish_errors = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.143988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] standard_threads = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.144972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_journal = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.145955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_json = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_stderr = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] use_syslog = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 19 12:16:55.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.147994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.148976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.149923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.150824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.151910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.152884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.153944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.154882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.155929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.156980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.157947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.158943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.159906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.160915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.161998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.162963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.163913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.164899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.165976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.166918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.167846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.168908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169749 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.169875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.170892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.171977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.external = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.172901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.password = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] auth.token = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.173991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.174961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.175950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.176981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.177950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.178920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.179994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.180994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.181956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.182877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.183956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.184970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.185952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.186915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.187991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.188991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.189109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.189708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.189829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.189949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.190962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.191916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.192930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.193892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.194959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.195926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.196914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.197877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.198971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.199972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.200941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.201898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.202930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.203993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.204111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.204267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.204388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.204505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.204638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.205458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.205587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.205706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.205824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.205952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.206982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.207944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.208928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.209894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.210955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.211891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.212993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.213951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.214899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.215977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.216948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.217902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.218980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.219963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.220986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.221937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.222898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.223979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 12:16:55.224097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.224097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.224097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.224097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 19 12:16:55.224097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80171) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 19 12:16:55.249801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.250036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.250266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.250351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.250556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.250783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.259141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.259505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.287316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.287433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.287565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.287647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.429389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.430037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.477682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.477797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.477933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.478040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.541815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.541939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.542899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.543022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.567792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.567903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.568933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.569016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.569151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.569240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.580708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.580847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.593980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.594718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80171) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 19 12:16:55.612150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.612384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 12:16:55.612627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.612720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.612819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 12:16:55.612898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 12:16:55.613139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.613223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.613340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 19 12:16:55.613423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.613609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.613693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.613868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.613950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.614126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.614381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.625159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.625435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.625701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.625781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.625983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.626076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.626253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.626533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.634648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.635192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.637928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.638205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.638477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.638558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.638758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.638853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.639987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.640193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.640277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.640464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.640547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.640729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.640809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.641001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.641269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.666940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.668157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.668559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.668838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.669792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.670075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.674577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.674852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.675131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.675867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.676144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.676448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.682404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.682646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.683696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.683981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.687226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.687509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.687785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.688062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.688359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.688606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.688690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.688881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.688962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.689144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.689267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.689971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.690246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.690521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.690787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.691052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.708242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.708509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.709713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.709990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.713169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.714625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.716798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.717088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.718275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.718571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.722111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.723619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.725784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.727243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.729549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.729843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.730131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.730420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.730694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.730965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.731054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.731310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.731601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.731893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.732206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.732479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.750470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.750839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.751105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.751365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.752095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.752494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.752767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.753033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.759331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.760952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.761038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.761305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.762641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.762917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.764343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.764618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.764890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.765162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.765446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.766018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.766296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.766575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.766853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.767180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.778723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.778997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.779273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.779570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.779851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.780133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.780432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.780515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.780778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.781061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.781344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.781623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.781911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.782182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.799963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.801847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.803236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.805138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.806527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.808115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.808469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.809746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.810020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.810101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.810301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.811690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.811969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.812284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.818218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.820023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.820240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.821975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.823288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.823574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.823853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.824130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.825213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.825501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.825781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.826051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.833379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.833665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.833938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.835296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.835581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.835859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.841719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80171) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.841994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.842268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.842539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.842806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.843074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.843349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.843620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.843708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 19 12:16:55.843899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.843981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.844979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.845063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.845240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80171) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.845322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.846075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.846356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.846627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.846896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.847155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.847419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.847683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 19 12:16:55.884172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564220e1e9c0 pid: 80171 (default app) Jul 19 12:16:55.886110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564220e1e9c0 pid: 80170 (default app) Jul 19 12:16:55.887444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032dd3-0117-40f6-94bf-c1dd06f7582d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:16:55.887739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032dd3-0117-40f6-94bf-c1dd06f7582d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:16:55.888023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032dd3-0117-40f6-94bf-c1dd06f7582d None None] PATH_INFO: `/v3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:16:55.892217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Tue Jul 19 12:16:55 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:03.566035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:03.566618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:03.566618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:03.571073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:03.576702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:03.577126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:03.577256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:03.648887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: INFO dbcounter [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] Registered counter for database keystone Jul 19 12:17:03.669939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG oslo_db.sqlalchemy.engines [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80171) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 12:17:03.676638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writer thread running {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 12:17:03.849920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG passlib.handlers.bcrypt [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80171) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 12:17:03.850269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG passlib.handlers.bcrypt [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80171) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 12:17:03.903886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:03.905518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:04.024664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 19 12:17:04.214705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:04.284429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] Registered counter for database keystone Jul 19 12:17:04.303144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 19 12:17:04.309290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 19 12:17:04.428009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb4cc9-d7a9-4101-8991-88de692fb415 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:04.432247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:04.432902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:04.433187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:04.433467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:04.442026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:04.493509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 12:17:04.493847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44994e5d-54f5-4b92-90cd-90524b19dbe2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:04.504108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Tue Jul 19 12:17:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 12:17:04.510046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:04.524426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8c58a-f874-4ac2-b67e-f8a58e7b3c90 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:04.528881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:04.529992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:04.530285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:04.530566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] PATH_INFO: `/v3/projects/45290f5162bc4ec4b133291838122f88` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:04.534652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] RBAC: Authorizing `identity:get_project(project_id=45290f5162bc4ec4b133291838122f88)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:04.585621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 19 12:17:04.586153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd5bd3-3d22-4102-a0ce-a97e272714c3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:04.590526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Tue Jul 19 12:17:04 2022] GET /identity/v3/projects/45290f5162bc4ec4b133291838122f88 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:06.575764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0560f-5aa4-4926-ac90-1eb92d623edc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:06.576308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0560f-5aa4-4926-ac90-1eb92d623edc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:06.576395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0560f-5aa4-4926-ac90-1eb92d623edc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:06.580519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:06.585838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:06.586251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:06.586409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:06.605298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:06.606835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:06.615427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 19 12:17:06.637324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0560f-5aa4-4926-ac90-1eb92d623edc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:06.649734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0560f-5aa4-4926-ac90-1eb92d623edc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:06.653868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fe66e-d280-4a13-95c7-9178187f8289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:06.654488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fe66e-d280-4a13-95c7-9178187f8289 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:06.654771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fe66e-d280-4a13-95c7-9178187f8289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:06.655046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fe66e-d280-4a13-95c7-9178187f8289 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:06.663392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fe66e-d280-4a13-95c7-9178187f8289 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:06.665201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fe66e-d280-4a13-95c7-9178187f8289 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:06.673970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Tue Jul 19 12:17:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 12:17:06.679110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:06.688023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725194bf-90e0-4fd6-bb74-92961732a220 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:06.692358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc8253-33a2-463c-91fb-7e2a98ddeab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:06.693213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8253-33a2-463c-91fb-7e2a98ddeab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:06.693500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8253-33a2-463c-91fb-7e2a98ddeab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:06.693780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc8253-33a2-463c-91fb-7e2a98ddeab3 admin admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:06.695167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc8253-33a2-463c-91fb-7e2a98ddeab3 admin admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:06.697044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc8253-33a2-463c-91fb-7e2a98ddeab3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:06.698591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Tue Jul 19 12:17:06 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:08.807749 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd91b0-46d9-4e72-b3e0-c1cb10b9caf9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:08.808324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd91b0-46d9-4e72-b3e0-c1cb10b9caf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:08.808413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd91b0-46d9-4e72-b3e0-c1cb10b9caf9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:08.809714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:08.815408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:08.815796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:08.815963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:08.822858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f7be-5022-4985-b546-a849e084ed02 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:08.822858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f7be-5022-4985-b546-a849e084ed02 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:08.822858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24f7be-5022-4985-b546-a849e084ed02 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:08.823776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:08.828573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:08.828862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:08.829145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:08.833584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:08.834748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:08.870786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 19 12:17:08.871133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 19 12:17:08.914991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:08.917011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:08.918585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:08.937202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:08.960889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.019225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:09.039811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441361c-0674-4b66-b153-0d3504eafb68 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.042406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.074908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.075563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.075850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.076136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.141653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2da8de-f4e1-475b-b156-26a9fb003684 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.142146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.143638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.143902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.166361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.178878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.179915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.180225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.180513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.192167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:09.204758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:09.205647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:09.206461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/14] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:09.211709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.246146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 7/15] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:17:09.252041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.309060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26376ece-4d15-41e3-8005-7a4b643b6655 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.341583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.342241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.342529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.342814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.343574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:09.343888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.345867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.347699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4cc45f-8181-4b77-b472-e6449c187e93 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.357386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:09.363431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.371339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.383096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.383885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.384168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.384476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.390207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.394554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.395977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:17:09.416940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:09.417944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:09.418727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:17:09.418727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:17:09.418727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:17:09.418727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:09.418727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:09.424401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.463493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2c18d-5ef8-4807-b899-3ed4179bf77d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.497277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.498143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.498429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.498704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.500285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.502264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.503869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:09.509239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.523657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401d76b-768c-4492-858e-7bd80ec43d46 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.557299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.557963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.558259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.558544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.559279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:09.559590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.561019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.578345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:09.606811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.616230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d47ff-da24-41d0-9645-a808f5d92f4b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.652668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7948401-4605-4095-b323-fea9005a1f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.653389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7948401-4605-4095-b323-fea9005a1f1b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.653677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7948401-4605-4095-b323-fea9005a1f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.653955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7948401-4605-4095-b323-fea9005a1f1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.655179 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7948401-4605-4095-b323-fea9005a1f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.656735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7948401-4605-4095-b323-fea9005a1f1b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.658788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:09.664993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:09.701680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:09.770293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccff66-75ca-4ff8-b05c-888c84f9da17 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:09.805000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:09.805665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:09.805954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:09.806236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:09.807075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.domain.id=default, user.id=f3447cd66131465b871daa6541b4276c) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:09.807392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:09.808929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddf8cd-ee67-4ef1-9561-3eb0059a665d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:09.816892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1322 bytes} [Tue Jul 19 12:17:09 2022] GET /identity/v3/role_assignments?user.id=f3447cd66131465b871daa6541b4276c&scope.domain.id=default&role.id=ac904f211a8f450fbf556c879296af4b => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:12.210142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056ef0-f548-4551-b7ce-b04811797830 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.211145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056ef0-f548-4551-b7ce-b04811797830 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.211654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0056ef0-f548-4551-b7ce-b04811797830 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.213283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:12.218750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05b9ac-5765-4efa-a8d1-c9000fd55d44 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.219560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05b9ac-5765-4efa-a8d1-c9000fd55d44 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.220113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05b9ac-5765-4efa-a8d1-c9000fd55d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.245436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1d05b9ac-5765-4efa-a8d1-c9000fd55d44 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:12.246977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1d05b9ac-5765-4efa-a8d1-c9000fd55d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:12.348114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:12.358011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d8756-f4bb-4261-ae1d-f2b2a5ce1c67 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.358580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d8756-f4bb-4261-ae1d-f2b2a5ce1c67 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.359073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d8756-f4bb-4261-ae1d-f2b2a5ce1c67 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.361235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:12.366547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.366846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.367134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.380543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d8756-f4bb-4261-ae1d-f2b2a5ce1c67 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:12.389284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:12.396204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:12.419325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:12.492038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 12/25] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:12.541326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:12.614196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d8756-f4bb-4261-ae1d-f2b2a5ce1c67 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:12.696444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ee016-5565-4731-907f-a8e46bad40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:12.697251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ee016-5565-4731-907f-a8e46bad40bf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.697545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ee016-5565-4731-907f-a8e46bad40bf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.697831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ee016-5565-4731-907f-a8e46bad40bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.716766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75265703-cc08-4635-a40b-b7683f04d627 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:12.730691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ee016-5565-4731-907f-a8e46bad40bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-be6ee016-5565-4731-907f-a8e46bad40bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:12.739136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:12.739985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:12.740758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:12.748672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/26] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:12.751540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c86bf8-28c9-4759-b199-0664f2824075 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.751953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c86bf8-28c9-4759-b199-0664f2824075 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.752426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c86bf8-28c9-4759-b199-0664f2824075 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.754241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/27] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:12.760302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c86bf8-28c9-4759-b199-0664f2824075 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:12.763693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f561-ac8b-4a1c-ab63-b34512b23ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:12.764406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f561-ac8b-4a1c-ab63-b34512b23ab3 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.764694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f561-ac8b-4a1c-ab63-b34512b23ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.764970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f561-ac8b-4a1c-ab63-b34512b23ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.765733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f561-ac8b-4a1c-ab63-b34512b23ab3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:12.767180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f561-ac8b-4a1c-ab63-b34512b23ab3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:12.807829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:12.905213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c86bf8-28c9-4759-b199-0664f2824075 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:12.937489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 13/28] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jul 19 12:17:12 2022] POST /identity/v3/roles => generated 225 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:12.939364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9384bf-28b6-4472-bcba-4932ab6b2a00 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.939857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9384bf-28b6-4472-bcba-4932ab6b2a00 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.940320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9384bf-28b6-4472-bcba-4932ab6b2a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.953667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:12.955558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.957130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.957574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.959145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:12.960558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:12.962179 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c444706-5546-43e7-84bf-85d59284737e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:12.983835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5e9384bf-28b6-4472-bcba-4932ab6b2a00 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:12.984438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/29] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:12.987133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8168-4461-4ab1-ad69-82a74526b5d3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:12.987873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5e9384bf-28b6-4472-bcba-4932ab6b2a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:12.988332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8168-4461-4ab1-ad69-82a74526b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:12.989888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc8168-4461-4ab1-ad69-82a74526b5d3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:12.992830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/30] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:12.993911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc8168-4461-4ab1-ad69-82a74526b5d3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.025872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.131150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 14/31] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:13.132710 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.133243 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.133678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.150985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:13.152540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:13.156624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.171008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc8168-4461-4ab1-ad69-82a74526b5d3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.217774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6fbfd-fafd-44cf-8b8f-2bd681986c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:13.218721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6fbfd-fafd-44cf-8b8f-2bd681986c53 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.218959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6fbfd-fafd-44cf-8b8f-2bd681986c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.219204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6fbfd-fafd-44cf-8b8f-2bd681986c53 None admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.221129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6fbfd-fafd-44cf-8b8f-2bd681986c53 None admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:13.222544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6fbfd-fafd-44cf-8b8f-2bd681986c53 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:13.224436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/32] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:12 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:13.229751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85319d-ed76-4d23-b515-fb62fe77f1b2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.230044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85319d-ed76-4d23-b515-fb62fe77f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.230356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85319d-ed76-4d23-b515-fb62fe77f1b2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.231607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/33] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:13.232653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85319d-ed76-4d23-b515-fb62fe77f1b2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.255611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 15/34] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:13.256339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.336314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.358508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6f874-80f5-4311-8fd0-5e46185850f4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.359684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85319d-ed76-4d23-b515-fb62fe77f1b2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.395051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:13.395850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.396140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.396452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.402039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb85e2-422d-4799-bdb3-02c6783735ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:13.403143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb85e2-422d-4799-bdb3-02c6783735ee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.403437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb85e2-422d-4799-bdb3-02c6783735ee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.403723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb85e2-422d-4799-bdb3-02c6783735ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.404971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb85e2-422d-4799-bdb3-02c6783735ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:13.406612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb85e2-422d-4799-bdb3-02c6783735ee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:13.409855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 16/35] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:13.411081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c692-5ca9-4408-a6c1-84ff1d1157c7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.411377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c692-5ca9-4408-a6c1-84ff1d1157c7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.411666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db26c692-5ca9-4408-a6c1-84ff1d1157c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.424908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:13.427251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:13.429165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-db26c692-5ca9-4408-a6c1-84ff1d1157c7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:13.430383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db26c692-5ca9-4408-a6c1-84ff1d1157c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:17:13.449271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:13.450272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:13.451100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:17:13.451100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:17:13.451100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:17:13.451100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:13.451100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/36] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:13.451346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.455292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.502628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 17/37] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:13.503940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879f60-2e60-412c-81e3-7b65a9bc5f0a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.504258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879f60-2e60-412c-81e3-7b65a9bc5f0a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.504539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879f60-2e60-412c-81e3-7b65a9bc5f0a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.508219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 18/38] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:13.508219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879f60-2e60-412c-81e3-7b65a9bc5f0a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.556359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9904e0-1dfb-4f6f-a445-6b291ffc2cc5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.582918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.590060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57073ca-09c4-4a37-aab6-ccac67e449b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:13.590660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57073ca-09c4-4a37-aab6-ccac67e449b9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.590945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57073ca-09c4-4a37-aab6-ccac67e449b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.591222 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57073ca-09c4-4a37-aab6-ccac67e449b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.591983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57073ca-09c4-4a37-aab6-ccac67e449b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:13.593404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57073ca-09c4-4a37-aab6-ccac67e449b9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:13.611733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879f60-2e60-412c-81e3-7b65a9bc5f0a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.647015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:13.647895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.648239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.648493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] PATH_INFO: `/v3/domains/default/users/f3447cd66131465b871daa6541b4276c/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.650482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f3447cd66131465b871daa6541b4276c, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:13.651977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:13.666195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 19/39] 10.222.0.13 () {58 vars in 1361 bytes} [Tue Jul 19 12:17:13 2022] PUT /identity/v3/domains/default/users/f3447cd66131465b871daa6541b4276c/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:13.667158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.714252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/40] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jul 19 12:17:13 2022] POST /identity/v3/roles => generated 219 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:13.715928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.716245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.716585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.749735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:13.752139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:13.826191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.840332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/41] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:13.840332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:13.922583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12c224-9f41-4ce1-a301-36be9eaf6779 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.961988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14206051-677b-45c9-ab5c-1002328b88c6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:13.970233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:13.971489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:13.971789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:13.972081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:13.974767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:13.975093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:13.977142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:13.983364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:13.996065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 20/42] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:13.997076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:14.004491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:14.005436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:14.005920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:14.006390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:14.007340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:14.008956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:14.022540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/43] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jul 19 12:17:13 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:14.023547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:14.102692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:14.129054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0720b-c6cb-4577-b5ea-04ddd292c094 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:14.156010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb98315-db0f-401c-b06b-cc390b49682c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:14.167510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:14.168106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:14.168435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:14.168720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:14.170138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:14.171885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:14.174180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 21/44] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:14.179831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:14.204444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05828a45-b162-4e66-93ba-f50b66bbaaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:14.205142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05828a45-b162-4e66-93ba-f50b66bbaaed None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:14.205477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05828a45-b162-4e66-93ba-f50b66bbaaed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:14.205763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05828a45-b162-4e66-93ba-f50b66bbaaed None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:14.206491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05828a45-b162-4e66-93ba-f50b66bbaaed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:14.207957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05828a45-b162-4e66-93ba-f50b66bbaaed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:14.245313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/45] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:17:14 2022] POST /identity/v3/projects => generated 301 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:14.258239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:14.307512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b06532-422c-4f36-ab73-fba0fc800313 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:14.344902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6dbcc-84e7-4d3c-a956-ce4d9ebf604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:14.345591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6dbcc-84e7-4d3c-a956-ce4d9ebf604a None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:14.345885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6dbcc-84e7-4d3c-a956-ce4d9ebf604a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:14.346170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6dbcc-84e7-4d3c-a956-ce4d9ebf604a None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:14.346906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6dbcc-84e7-4d3c-a956-ce4d9ebf604a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:14.348445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6dbcc-84e7-4d3c-a956-ce4d9ebf604a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:14.384301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 22/46] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:17:14 2022] POST /identity/v3/projects => generated 312 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:15.774875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461996f0-7fa6-41f5-8297-3f2d9b29e541 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:15.775375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461996f0-7fa6-41f5-8297-3f2d9b29e541 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:15.775457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461996f0-7fa6-41f5-8297-3f2d9b29e541 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:15.776822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/47] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:15.782086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:15.782469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:15.782654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:15.799648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:15.800842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:15.865422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 23/48] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:15.888726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461996f0-7fa6-41f5-8297-3f2d9b29e541 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:15.986751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461996f0-7fa6-41f5-8297-3f2d9b29e541 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:15.993897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:16.021481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.022276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.022570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.022853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.033284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:16.041926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:16.042988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:16.044045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/49] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:16.049620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:16.096896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:16.149303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f44c8f-56c1-4363-aa46-f9a8cf0f119c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:16.182357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.183026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.183312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.183591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.184402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:16.184710 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:16.186107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:16.202892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 24/50] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:16.208534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:16.305044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1bb8b-b6ac-4389-8fcf-92f800ddbbed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:16.339126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10fb37-07ad-474d-bc87-ac43edd629e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.340001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10fb37-07ad-474d-bc87-ac43edd629e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.340329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10fb37-07ad-474d-bc87-ac43edd629e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.340616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10fb37-07ad-474d-bc87-ac43edd629e4 None admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.341818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10fb37-07ad-474d-bc87-ac43edd629e4 None admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:16.343278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10fb37-07ad-474d-bc87-ac43edd629e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:16.345819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:16 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:16.351445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:16.436158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dbee-1b23-448e-a8b5-2da7ec2c2409 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.436397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dbee-1b23-448e-a8b5-2da7ec2c2409 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.436690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90dbee-1b23-448e-a8b5-2da7ec2c2409 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.438066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/52] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:16.442998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.443293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.443577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.452226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83353bc-b815-465d-89f3-d3e5985e7ed8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:16.460065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:16.461275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:16.461876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:16.486881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.487672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.487966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.488441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.488553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:16.489511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:16.491348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:16.492994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 25/53] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:16.499025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:16.529592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/54] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:16.552394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:16.598632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e24be-11f8-4bce-89b2-5cc1a1ed8f94 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:16.633620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.634281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.634575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.634857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.635696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.domain.id=default, user.id=f3447cd66131465b871daa6541b4276c) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:16.636013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:16.637508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:16.644054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 26/55] 10.222.0.13 () {56 vars in 1322 bytes} [Tue Jul 19 12:17:16 2022] GET /identity/v3/role_assignments?user.id=f3447cd66131465b871daa6541b4276c&scope.domain.id=default&role.id=ac904f211a8f450fbf556c879296af4b => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:16.650160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23214cf7-7768-4476-9de4-ecea11d76bcf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:16.687009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0261bd-9492-41c8-9b8c-8c71c1565cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.687736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0261bd-9492-41c8-9b8c-8c71c1565cfa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.688022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0261bd-9492-41c8-9b8c-8c71c1565cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.688922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0261bd-9492-41c8-9b8c-8c71c1565cfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.690210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0261bd-9492-41c8-9b8c-8c71c1565cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:16.691713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0261bd-9492-41c8-9b8c-8c71c1565cfa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:16.693467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:16.699437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:16.743275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:16.804985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38450a8d-e79a-495c-91ae-9ca0611e4c01 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:16.837851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9453af-635e-4b28-89e4-4b51da96ea1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:16.838524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9453af-635e-4b28-89e4-4b51da96ea1e None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:16.838810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9453af-635e-4b28-89e4-4b51da96ea1e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:16.839093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9453af-635e-4b28-89e4-4b51da96ea1e None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:16.839825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9453af-635e-4b28-89e4-4b51da96ea1e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:16.841311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9453af-635e-4b28-89e4-4b51da96ea1e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:16.868352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 27/57] 10.222.0.13 () {60 vars in 1140 bytes} [Tue Jul 19 12:17:16 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:18.906078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b23ff-e7a6-411d-b5ee-b0cc1391e053 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:18.906562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b23ff-e7a6-411d-b5ee-b0cc1391e053 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:18.906562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b23ff-e7a6-411d-b5ee-b0cc1391e053 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:18.907956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:18.913021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:18.913394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:18.913595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:18.930409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:18.931615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:18.996633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:19.020585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b23ff-e7a6-411d-b5ee-b0cc1391e053 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:19.056907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:19.126367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b23ff-e7a6-411d-b5ee-b0cc1391e053 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:19.158962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bac8c-8aba-4feb-81da-d6784857d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:19.159675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bac8c-8aba-4feb-81da-d6784857d52c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:19.159959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bac8c-8aba-4feb-81da-d6784857d52c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:19.160262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bac8c-8aba-4feb-81da-d6784857d52c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:19.161419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bac8c-8aba-4feb-81da-d6784857d52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:19.162862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bac8c-8aba-4feb-81da-d6784857d52c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:19.164474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:19.170014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:19.193914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:19.267889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f276a57-ae4e-467a-9357-95558fcd4cf9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:19.301281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23eba9-11c4-41d2-9805-159ac42560c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:19.301956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23eba9-11c4-41d2-9805-159ac42560c5 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:19.302254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23eba9-11c4-41d2-9805-159ac42560c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:19.302537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23eba9-11c4-41d2-9805-159ac42560c5 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:19.303270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23eba9-11c4-41d2-9805-159ac42560c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:19.304797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23eba9-11c4-41d2-9805-159ac42560c5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:19.332797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:19 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:22.247974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a7ac-d1a3-4545-a312-48b1e08430ba None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.248512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a7ac-d1a3-4545-a312-48b1e08430ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.248615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9a7ac-d1a3-4545-a312-48b1e08430ba None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.250098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:22.266246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.266742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.266815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.289280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:22.290497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:22.382520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 30/63] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:22.420303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a7ac-d1a3-4545-a312-48b1e08430ba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:22.631088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:22.635311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9a7ac-d1a3-4545-a312-48b1e08430ba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:22.675822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab732-804d-4f1c-9d10-6b80e568ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:22.676661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab732-804d-4f1c-9d10-6b80e568ced7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.676959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab732-804d-4f1c-9d10-6b80e568ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.677243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab732-804d-4f1c-9d10-6b80e568ced7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.690789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ab732-804d-4f1c-9d10-6b80e568ced7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9c3ab732-804d-4f1c-9d10-6b80e568ced7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:22.698623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:22.699455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:22.700211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/64] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:22.706558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:22.728597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:22.742401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39426448-a80e-46af-a14c-4ec36a245d71 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.742766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39426448-a80e-46af-a14c-4ec36a245d71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.742975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39426448-a80e-46af-a14c-4ec36a245d71 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.744779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/65] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:22.749713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4635-7676-4c07-bbbf-741832eac124 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.750008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4635-7676-4c07-bbbf-741832eac124 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.750523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4635-7676-4c07-bbbf-741832eac124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.782799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7aaf4635-7676-4c07-bbbf-741832eac124 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:22.784015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7aaf4635-7676-4c07-bbbf-741832eac124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:22.848097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd1db9-a422-4d0a-97c8-5ef69685648e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:22.869072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/66] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:22.870605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e444c-a5cc-4b9d-b369-56bb0bfa603d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.870893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e444c-a5cc-4b9d-b369-56bb0bfa603d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.871171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e444c-a5cc-4b9d-b369-56bb0bfa603d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.880885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/67] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:22.884957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6c4e5-5ef8-4db6-b772-3d645f700dc5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.885250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6c4e5-5ef8-4db6-b772-3d645f700dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.885535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6c4e5-5ef8-4db6-b772-3d645f700dc5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.887059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/68] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:22.887815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:22.888600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.888897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.889186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.889956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:22.890270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:22.891929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512545da-3aef-47ee-a840-7fd6ca02b4fd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:22.892160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.892474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.892761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.902912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 31/69] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:22.904332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:22.904628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:22.904956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:22.910790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:22.911965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:22.922201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:22.923432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:22.980781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/70] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:22.981639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:22.996506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.003199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 32/71] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:23.009479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.012587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.102149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465c612-f6e3-4f75-8ae2-d6713599ba95 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.120029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0fec5-9842-442e-baa2-c881f44b9cec None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.145767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.146601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.147201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.147485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.158081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.159241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.159853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:23.160078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.160565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:23.168944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:23.169766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/72] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:23.176142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.183869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.185819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:23.188766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 33/73] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:23.189836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.250219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.275301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.298335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deacd0-9888-4f94-8d1e-37ebc2936131 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.304895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b52c1-32a9-4da9-a247-a277a178f819 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.334727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.335549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.335839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.336120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.349417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.349682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:23.350292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.350590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.350869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:23.359074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:23.360523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:23.361451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 34/74] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:23.362076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.366791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:23.382536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:23.383487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:23.384272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:23.384898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/75] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:23.385560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.478283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59308af-42c0-4f05-a3ac-898268050c9b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.496237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.514347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fec60-86a0-4213-a515-f6b37b83e3f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.518760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.519442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.519734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.520021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.520824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:23.521141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.522550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b9a5a-6668-4e36-a906-7dae64be1bfd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:23.531156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 35/76] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:23.532965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211feea0-3d7f-4917-ad09-4cdb01691cc6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.533263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211feea0-3d7f-4917-ad09-4cdb01691cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.533548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211feea0-3d7f-4917-ad09-4cdb01691cc6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.534954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 36/77] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:23.539551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211feea0-3d7f-4917-ad09-4cdb01691cc6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.549296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.549941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.550230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.550511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.561086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.619533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.621068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:23.642371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211feea0-3d7f-4917-ad09-4cdb01691cc6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.674534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.675200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.675482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.675766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.676542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:23.676854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.678230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:23.685851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 37/78] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:23.686631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.719807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/79] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:23.720830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.732835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.763264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.790666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1395a65-37a3-42fc-b5ae-0b4392c53b4b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.821518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce8979-d892-46be-a8c3-207c9a36d943 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.824044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.824746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.825040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.825323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.826070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:23.826388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.827771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b5f22-017b-4604-a452-35a4e75a0d82 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:23.836303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 38/80] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:23.841457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65860be6-02f7-49b1-a729-395017f01377 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.842165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65860be6-02f7-49b1-a729-395017f01377 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.842456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65860be6-02f7-49b1-a729-395017f01377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.855082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.855954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.856282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.856568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.857766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.858099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-65860be6-02f7-49b1-a729-395017f01377 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:23.859209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:23.859307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-65860be6-02f7-49b1-a729-395017f01377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:23.860915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/81] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:23.861672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.925767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 39/82] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:23.926516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65860be6-02f7-49b1-a729-395017f01377 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:23.960743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65046062-9308-49d0-a8e9-1f19ad92403f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:23.969136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.970030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:23.995247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:23.996217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:23.996519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:23.996805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:23.998022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:23.999461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.001670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/83] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:24.002436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.039639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65860be6-02f7-49b1-a729-395017f01377 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.074423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.075285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.075577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.075863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.077150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.078642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.080402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 40/84] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:23 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:24.081175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.102701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee980ad-4136-4f8c-aa76-82c0f38e153b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.136841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.137704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.138000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.138287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.139501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.141050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.142688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/85] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:24.143459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.180005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9414b-9615-40fd-bde9-5f9d68167c20 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.181955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.187633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.216252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.216906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.217199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.217481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.242185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254bc0f-28d3-49d7-a638-04ccc8f8cd2c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.277111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.277891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.278187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.278474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.279665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.281264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.283625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/86] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:24.284221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.288892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.290418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.381617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018231fc-da20-4f62-98dd-98cc1f5d551d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.387718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 41/87] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:24.388426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.390591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.396416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.418050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.418700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.418986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.419264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.486732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.488352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.489883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4c73c-eb02-4959-9049-04ea35ff4a2f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.522963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.523614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.523901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.524264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.525045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:24.525361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.526798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.532631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 42/88] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/role_assignments?user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=13dc310ce9ce466088231180b4040e69 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:24.533402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.603716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.614460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/89] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:24.619743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.631596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.658205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2972b-31ba-4238-aba3-9c90052735f4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.703782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.704475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.704769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.705053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.732702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332058e-722c-4912-81c3-7fad65464bd2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.769807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.770394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.770679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.770956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.771675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.773163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.776907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.778369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:24.808232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/90] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:24 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:24.808232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.853289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.896087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:24.897115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 43/91] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:24.897945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:24.953572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a80a-30a1-4e70-973d-4095df27efb2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:24.994585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:24.995972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:24.996395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:24.996676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:24.997901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:24.999536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:25.002174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/92] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:25.002959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:25.019964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1c027-61b2-433f-bcc0-251f88aa6051 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:25.060972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:25.061854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:25.062145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:25.062423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:25.064169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:25.065680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:25.067564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 44/93] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:24 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:25.072334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:25.115016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:25.130552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a357-d1ee-4d3c-b840-38f07c08a46e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:25.172827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:25.174254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:25.174254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:25.174254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] PATH_INFO: `/v3/projects/6f8edb3ba7a5427996b2b75fa2106439` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:25.175864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] RBAC: Authorizing `identity:get_project(project_id=6f8edb3ba7a5427996b2b75fa2106439)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:25.178043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:25.180079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:25 2022] GET /identity/v3/projects/6f8edb3ba7a5427996b2b75fa2106439 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:25.184790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:25.191450 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:25.205233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f06b8-4833-4006-bf02-4abe0c4e7632 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:25.246208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:25.246939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:25.247235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:25.247517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:25.248394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=f3447cd66131465b871daa6541b4276c) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:25.248719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:25.250847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:25.257038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 45/95] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:25 2022] GET /identity/v3/role_assignments?user.id=f3447cd66131465b871daa6541b4276c&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:25.257979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:25.348146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb25c5-1784-43eb-888b-d69c5833751e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:25.402809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:25.403956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:25.404470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:25.405043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:25.406081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] RBAC: Adding query filter params (role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:25.406649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:25.414595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a9a8f-e4b9-4972-a592-cd6237c66b07 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:25.433359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:25 2022] GET /identity/v3/role_assignments?user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=7805367da7144187ae1dd2e09705a569 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:25.435268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:25.441324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cad10-fbba-46f9-bc0e-1904ce17ee13 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:25.480754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:25.481431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:25.481721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:25.482006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:25.482857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=6f8edb3ba7a5427996b2b75fa2106439, user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:25.483183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:25.485238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52dccb-20cb-4b4a-bc7b-bd237175be46 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:25.491787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:25 2022] GET /identity/v3/role_assignments?user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3&scope.project.id=6f8edb3ba7a5427996b2b75fa2106439&role.id=13dc310ce9ce466088231180b4040e69 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:26.806183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2479536-fccc-4e38-a0c6-4b4308f05dc9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:26.806692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2479536-fccc-4e38-a0c6-4b4308f05dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:26.806768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2479536-fccc-4e38-a0c6-4b4308f05dc9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:26.808116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:26.813613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:26.814012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:26.814196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:26.837203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:26.838428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:26.914971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 47/99] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:26.937902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2479536-fccc-4e38-a0c6-4b4308f05dc9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:27.015357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:27.058774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2479536-fccc-4e38-a0c6-4b4308f05dc9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:27.105003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:27.105798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.106088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.106372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.118481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:27.126712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:27.127542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:27.129622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/100] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:27.136200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:27.225283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:27.233806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24286a-d2b3-407c-8b31-3a8ea6dedc14 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:27.269251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:27.269930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.270223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.270508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.271265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:27.271577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:27.273005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:27.282084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 48/101] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:27.287715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:27.401485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0189ed-95f2-4570-a8f1-a0522cabd56c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:27.437766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:27.446676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:27.447605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.447950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.448259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.452944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:27.454469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:27.456244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/102] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:27 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:27.461853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:27.566772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274ea40-8e18-4633-9a15-15c7bcbfcaef None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:27.601451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40f78-a4f2-4be4-9148-76d8bf270b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:27.602314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40f78-a4f2-4be4-9148-76d8bf270b8a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.602599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40f78-a4f2-4be4-9148-76d8bf270b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.602894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40f78-a4f2-4be4-9148-76d8bf270b8a None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.604107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a40f78-a4f2-4be4-9148-76d8bf270b8a None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:27.605636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a40f78-a4f2-4be4-9148-76d8bf270b8a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:27.607370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 49/103] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:27 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:27.613144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:27.743505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98a70c-517d-475c-9f54-a0b621787001 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:27.790038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:27.790943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.791238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.791521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.796068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2df5ade771f42f78fe0ab403129aa5d, user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:27.798111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:27.805625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11bfa-1c2e-4253-8b0f-259e8924e146 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.806212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11bfa-1c2e-4253-8b0f-259e8924e146 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.806712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11bfa-1c2e-4253-8b0f-259e8924e146 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.808313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 50/104] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:27.814860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:27.815944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:27.816600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:27.833297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/105] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:27 2022] PUT /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:27.837148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:27.838795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:27.858088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:27.921762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 51/106] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:27.960497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:28.044228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:28.099029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c115289-7e6b-446b-a7af-afc7a7dc4944 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:28.137718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:28.138609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.138897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.139178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.150672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:28.158432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:28.159305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:28.160062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/107] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:28.165444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:28.271442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ee9b7-e342-4175-a482-9dc0b1b8f872 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:28.306676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:28.307344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.307624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.307906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.308686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:28.308995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:28.310388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614093d-30ee-47c1-bb1a-e3291d270706 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:28.319572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 52/108] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:28.325238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:28.351329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1e13-e714-4f4c-9f19-6fe7d51faf7a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.351619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1e13-e714-4f4c-9f19-6fe7d51faf7a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.351936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d1e13-e714-4f4c-9f19-6fe7d51faf7a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.353486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 53/109] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:28.364037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f137-56a8-4d65-9f11-7cb524948d27 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.364358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f137-56a8-4d65-9f11-7cb524948d27 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.364644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f137-56a8-4d65-9f11-7cb524948d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.393170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8d78f137-56a8-4d65-9f11-7cb524948d27 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:28.394399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8d78f137-56a8-4d65-9f11-7cb524948d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:28.414740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:28.551551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 54/110] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:28.552940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de997c6-a7a0-44c5-8ea6-15e2dd56cae4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.553234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de997c6-a7a0-44c5-8ea6-15e2dd56cae4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.553521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de997c6-a7a0-44c5-8ea6-15e2dd56cae4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.555134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 55/111] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:28.560248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.560544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.560831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.567028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59757d-9c27-400b-81e1-cb6c241447af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:28.589961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:28.591144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:28.606004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:28.640842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:28.641780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.642075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.642359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] PATH_INFO: `/v3/users/49a7845a07424dc8a3b526cae02fb738` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.678599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] RBAC: Authorizing `identity:get_user(user_id=49a7845a07424dc8a3b526cae02fb738)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:28.680098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:28.681872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/112] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:28 2022] GET /identity/v3/users/49a7845a07424dc8a3b526cae02fb738 => generated 289 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:28.686467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:28.722690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 56/113] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:28.723705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:28.808596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:28.831742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747d7a-97b0-4355-a326-ab59a613f2c7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:28.884062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e8e6d-9a27-4537-910f-5a9c0be478ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:28.884999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8e6d-9a27-4537-910f-5a9c0be478ea None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.885297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8e6d-9a27-4537-910f-5a9c0be478ea None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.885583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8e6d-9a27-4537-910f-5a9c0be478ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.919047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d1649-19fd-4cd6-82a2-686a4354838e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:28.966144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:28.967226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:28.967333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:28.967679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:28.975046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e8e6d-9a27-4537-910f-5a9c0be478ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:28.976814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e8e6d-9a27-4537-910f-5a9c0be478ea None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:28.978088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:28.985923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:28.986759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:28.987508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:28.988116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.033951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.071869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.110013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 57/115] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:29.111735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8d62d-9a74-4772-9b83-5ce550c02030 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.112236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8d62d-9a74-4772-9b83-5ce550c02030 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.112906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8d62d-9a74-4772-9b83-5ce550c02030 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.114555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 58/116] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:29.115605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8d62d-9a74-4772-9b83-5ce550c02030 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.121785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00434a-c787-4d18-97c7-c2102750a602 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.156626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.157337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.157624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.157905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.159609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.161103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.162720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/117] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:29.163486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.234994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8d62d-9a74-4772-9b83-5ce550c02030 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.273661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bd3fb-05a8-4d6f-ba7a-6eb266d08f7c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.275430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.277513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.278468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.278961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.279448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.280593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:29.281506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.282964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbba96-c227-4b6b-a860-9bfce129e7cd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.298934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 59/118] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:29.298934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381734-e788-4e82-9efc-86d0f7f16c43 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.298934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381734-e788-4e82-9efc-86d0f7f16c43 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.298934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381734-e788-4e82-9efc-86d0f7f16c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.311804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.313285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.313578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.313856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.315277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.315911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ad381734-e788-4e82-9efc-86d0f7f16c43 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:29.316856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.317581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ad381734-e788-4e82-9efc-86d0f7f16c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:29.323687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/119] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:29.323687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.353413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.396045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 60/120] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:29.397706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bf9ad-eb39-4f08-a207-586094aeacbe None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bf9ad-eb39-4f08-a207-586094aeacbe None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.398426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bf9ad-eb39-4f08-a207-586094aeacbe None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.399927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 61/121] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:29.400607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bf9ad-eb39-4f08-a207-586094aeacbe None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.443144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dae579-7420-470f-ae92-6c62f17cdba5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.481049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.481713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.482006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.482321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.483067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.485092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.520216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/122] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:17:29 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:29.520216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.522803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bf9ad-eb39-4f08-a207-586094aeacbe None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.549740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.573435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9c924-42b1-4c08-8b2f-19f5c168aba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.574978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c924-42b1-4c08-8b2f-19f5c168aba4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.575343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c924-42b1-4c08-8b2f-19f5c168aba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.575742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9c924-42b1-4c08-8b2f-19f5c168aba4 None admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.577943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c924-42b1-4c08-8b2f-19f5c168aba4 None admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.580521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9c924-42b1-4c08-8b2f-19f5c168aba4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.582746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 62/123] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:29.585591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.585882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.586160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.610276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:29.611722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:29.651919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.653110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425673a-7a12-4741-8ef9-a253402d721f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.698141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.699241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.699241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.699429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.700202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] RBAC: Adding query filter params (role.id=f74f6ab86814400fb36f9d8a394c9a90, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=49a7845a07424dc8a3b526cae02fb738) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:29.700459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.702180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.710074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/124] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/role_assignments?user.id=49a7845a07424dc8a3b526cae02fb738&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=f74f6ab86814400fb36f9d8a394c9a90 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:29.711158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.720306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 63/125] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:29.720306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.852382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:29.887495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead080c-c608-4853-b1ee-a204e6e02a82 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.895219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83480-6df5-4a60-b671-9574e161ef9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:29.954821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.955628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.955921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.956225 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.958737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:29.960459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:29.961280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:29.961573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:29.961857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:29.962878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:29.964804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 64/126] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:29.965669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:29.992800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:30.001424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:30.002322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:30.003099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:30.008644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/127] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:30.009504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.012564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:30.090162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb221741-3228-4b62-a7ee-482322326e31 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.133374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.134554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.135049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.135560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.147363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:30.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:30.157026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:30.157766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:30.160359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 65/128] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:30.161380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.231238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4955d1-1ddc-4f47-8672-eb2069d31b73 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.272626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:30.273185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.274575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.274906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.275260 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/f3447cd66131465b871daa6541b4276c/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.278449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2df5ade771f42f78fe0ab403129aa5d, user_id=f3447cd66131465b871daa6541b4276c, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:30.280104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:30.283344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163035b-17f4-4699-9c4b-cf46eaf69503 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.295598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/129] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:30 2022] PUT /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/f3447cd66131465b871daa6541b4276c/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:30.296614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.307032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:30.345253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.346636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.346636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.346636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.347283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:30.347598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:30.349050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db3841-9a1a-49c0-9132-1d1687fb9178 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:30.370177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 66/130] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:30.370177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f69b2-dddf-4452-991f-c51b602f9b58 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.370177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f69b2-dddf-4452-991f-c51b602f9b58 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.370177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f69b2-dddf-4452-991f-c51b602f9b58 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.370177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 67/131] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:30.377675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.377966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.378250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.411822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:30.413084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:30.429977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94334dc-1ae6-4aa2-bad5-1d3240163df2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.473510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.474955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.475842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.476897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.478249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:30.479779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:30.481757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:30.503523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/132] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:30.506533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.551084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 68/133] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:30.552249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.628660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:30.653649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:30.661743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb1d63a-d454-4ae4-b30b-d5fa37aa7970 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.671254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc878303-87e3-4c0c-86ec-133d5c6fb74f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.697268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.698242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.698533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.698814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.700115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:30.701775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:30.704185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/134] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:30.705066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.710256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.711146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.711435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.711716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.713131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:30.714858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:30.716713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 69/135] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:30.717479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.832990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715111d2-b61f-4fd8-9430-993edc687662 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.875460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d893cca-51dd-4d8f-bdb2-c8662beff399 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:30.877363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.878163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.878448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.878725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.884136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:30.905127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:30.920207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:30.921093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:30.922373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:30.923584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/136] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:30.924138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.935464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:30.936464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:30.936746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:30.937023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:30.939230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:30.940727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:30.942529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 70/137] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:30.943304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:30.998634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:31.113704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5617c-08a8-4faf-aa99-a1a9419de995 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:31.139904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4138050-747d-4d78-9db9-56ff285de55b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:31.155531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:31.157250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:31.157750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:31.158264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] PATH_INFO: `/v3/projects/6f8edb3ba7a5427996b2b75fa2106439` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:31.159860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] RBAC: Authorizing `identity:get_project(project_id=6f8edb3ba7a5427996b2b75fa2106439)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:31.163269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:31.165450 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/138] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/projects/6f8edb3ba7a5427996b2b75fa2106439 => generated 312 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:31.167816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:31.212877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:31.213561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:31.213849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:31.214129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:31.214895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:31.215202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:31.216632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:31.231597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 71/139] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:31.232405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:31.241486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:31.289274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e36ed-d73b-41b6-8626-e4544b009267 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:31.340918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:31.342194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:31.342716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:31.343200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:31.349516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] RBAC: Authorizing `identity:create_grant(project_id=d2df5ade771f42f78fe0ab403129aa5d, user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3, role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:31.351262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:31.362186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:31.379632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ccae8c-9f97-4bcd-a62e-4c4a919bde8c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:31.386374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/140] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:31 2022] PUT /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3/roles/7805367da7144187ae1dd2e09705a569 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:31.387579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:31.419815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:31.421061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:31.421355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:31.421681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] PATH_INFO: `/v3/projects/6f8edb3ba7a5427996b2b75fa2106439/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:31.424938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f8edb3ba7a5427996b2b75fa2106439, user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:31.427163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:31.443049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 72/141] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:31 2022] PUT /identity/v3/projects/6f8edb3ba7a5427996b2b75fa2106439/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:31.541523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1434-ec26-43f5-8a12-7769e01669ff None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:31.572572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:31.611860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:31.612889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:31.613193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:31.613480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:31.614975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:31.616482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:31.618393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/142] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:31 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:31.625413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:31.773615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abe490-cd25-46eb-9562-eda1b3a69893 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:31.812159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:31.813372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:31.813658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:31.813938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:31.827919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:31.889964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:31.891524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:32.020240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 73/143] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:32.025495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:32.152914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6ab8d-2468-4a83-8418-45b14ce3f2e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:32.200891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87c31f-8506-4728-87b3-4cd052c8a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:32.202130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87c31f-8506-4728-87b3-4cd052c8a363 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.202644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87c31f-8506-4728-87b3-4cd052c8a363 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.203774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87c31f-8506-4728-87b3-4cd052c8a363 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.205381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87c31f-8506-4728-87b3-4cd052c8a363 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:32.207043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87c31f-8506-4728-87b3-4cd052c8a363 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:32.209025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/144] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:32 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:32.217315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:32.300486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:32.373282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96aefa-d225-449e-a88c-41291a034252 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:32.414144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abc98e-4785-4854-b943-16126cafc298 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.414797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abc98e-4785-4854-b943-16126cafc298 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.415284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abc98e-4785-4854-b943-16126cafc298 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.416908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/145] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:32.423321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.423941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.424464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.424565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:32.425131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.425417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.425697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.426535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:32.426849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:32.436214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19d9cf-a259-4e0c-83a6-1513c33bff43 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:32.439849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 74/146] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:32 2022] GET /identity/v3/role_assignments?user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=13dc310ce9ce466088231180b4040e69 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:32.454494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c1b4-968c-425f-9217-4bec51e0725a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.454792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c1b4-968c-425f-9217-4bec51e0725a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.455066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c1b4-968c-425f-9217-4bec51e0725a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.457984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 75/147] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:32.466224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:32.467477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:32.470329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.470617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.470896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.501581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:32.502809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:32.584636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:32.607380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/148] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:32.625884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 76/149] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:32.648885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:32.661651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:32.725253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:32.803600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083518d1-2c8b-4457-9f30-4c1305a04116 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:32.850918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822af7ec-4a4c-4654-8320-8a7a43421fc8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:32.872910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:32.873722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.874009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.874288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.891011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:32.891192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:32.892265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:32.892754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:32.893257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:32.894640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:32.896404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:32.898239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 77/150] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:32.902175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:32.903015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:32.903766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:32.909840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/151] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:32.911293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:32.917078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:32.986863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:33.051840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dadd1-c963-4c19-9b5e-2c2b80a04a6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:33.054266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120b100-b066-49a3-b3c6-fb3457bf1d2e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:33.061656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:33.095227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:33.095897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.096203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.096494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.097252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:33.097563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:33.098969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:33.107729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/152] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:33.114513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:33.118225 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:33.118916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.119449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.119725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.120482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:33.121961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:33.158687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 78/153] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:32 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:33.262164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c8581-651c-48bc-bebf-72b7036977a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:33.278495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:33.303632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d70ea2-3dfb-4987-94b1-be8313fb77fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:33.304535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d70ea2-3dfb-4987-94b1-be8313fb77fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.304825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d70ea2-3dfb-4987-94b1-be8313fb77fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.305106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d70ea2-3dfb-4987-94b1-be8313fb77fd None admin] PATH_INFO: `/v3/users/49a7845a07424dc8a3b526cae02fb738` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.308957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d70ea2-3dfb-4987-94b1-be8313fb77fd None admin] RBAC: Authorizing `identity:get_user(user_id=49a7845a07424dc8a3b526cae02fb738)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:33.310437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d70ea2-3dfb-4987-94b1-be8313fb77fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:33.312139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/154] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:33 2022] GET /identity/v3/users/49a7845a07424dc8a3b526cae02fb738 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:33.318672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:33.411443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9341b40-271e-4372-8609-a1afdf79a0a5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.411734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9341b40-271e-4372-8609-a1afdf79a0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.411930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9341b40-271e-4372-8609-a1afdf79a0a5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.413500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/155] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:33.419014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.419295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.419594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.444299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:33.445544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:33.471913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a34a5-9449-41a0-bc36-0cadd8de5d03 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:33.529662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:33.530544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.530831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.531634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.533113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:33.534586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:33.536761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 79/156] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:33 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:33.542247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:33.556320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/157] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 19 12:17:33.583316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:33.585226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:33.679117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758d2a6-471e-4871-b7f5-15fafd95da55 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:33.723936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:33.725342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.725870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.726324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/49a7845a07424dc8a3b526cae02fb738/roles/f74f6ab86814400fb36f9d8a394c9a90` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.730296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] RBAC: Authorizing `identity:create_grant(project_id=d2df5ade771f42f78fe0ab403129aa5d, user_id=49a7845a07424dc8a3b526cae02fb738, role_id=f74f6ab86814400fb36f9d8a394c9a90)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:33.732407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:33.736548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233681b5-1558-4003-a42f-e00890bfec14 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:33.755477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:33.758872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 80/158] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:33 2022] PUT /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/users/49a7845a07424dc8a3b526cae02fb738/roles/f74f6ab86814400fb36f9d8a394c9a90 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:17:33.794331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:33.795395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:33.795832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:33.796287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:33.811574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:33.820087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:33.821321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:33.822296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:33.825116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/159] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:33.831442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:33.956804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:33.979930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ebffc-ac83-472c-a773-2a3ffe06b80f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:34.018320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:34.018999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.019280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.019625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.020419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:34.020727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:34.022175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:34.036309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 81/160] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:34.037877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.146634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2052e21-fc18-4c93-88a4-e62fe76fd6ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:34.181917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1513463-d24b-41a8-8dc8-0b772b6e7e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:34.182895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1513463-d24b-41a8-8dc8-0b772b6e7e45 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.183185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1513463-d24b-41a8-8dc8-0b772b6e7e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.183463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1513463-d24b-41a8-8dc8-0b772b6e7e45 None admin] PATH_INFO: `/v3/users/f3447cd66131465b871daa6541b4276c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.184695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1513463-d24b-41a8-8dc8-0b772b6e7e45 None admin] RBAC: Authorizing `identity:get_user(user_id=f3447cd66131465b871daa6541b4276c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:34.186149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1513463-d24b-41a8-8dc8-0b772b6e7e45 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:34.188640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/161] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/users/f3447cd66131465b871daa6541b4276c => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:34.194205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.245262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15326-1c16-48a4-a190-9c6163211957 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.245524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15326-1c16-48a4-a190-9c6163211957 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.245831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15326-1c16-48a4-a190-9c6163211957 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.247290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/162] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:34.257296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.257619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.257898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.279573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:34.282123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:34.327174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:34.341501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e20c6-c18d-4976-ac3a-168f167a437c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:34.384701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/163] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:34.398193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:34.398851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.399134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.399415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.422241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.428596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:34.482962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:34.485440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:34.533426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d05319-e3e0-40fb-b2f2-8c4d43745221 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:34.571455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db1833-0315-4b8f-96ef-4b4c522964c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:34.572389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db1833-0315-4b8f-96ef-4b4c522964c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.572775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db1833-0315-4b8f-96ef-4b4c522964c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.573052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db1833-0315-4b8f-96ef-4b4c522964c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.585432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db1833-0315-4b8f-96ef-4b4c522964c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f6db1833-0315-4b8f-96ef-4b4c522964c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:34.593648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:34.594535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/164] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:34.597015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b86b27-d155-4969-b8c9-4192f112ea38 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.598004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b86b27-d155-4969-b8c9-4192f112ea38 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.598004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b86b27-d155-4969-b8c9-4192f112ea38 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.598329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/165] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:34.603587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.603884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.611300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.616032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 82/166] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:34.619338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.630358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:34.631546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:34.656103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:34.688662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:34.704377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/167] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:34.705058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.773018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be97e9-d9c1-4de7-88dc-6009062ef890 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:34.831021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8e345-e29a-44ae-bc7f-2d9fcc3ba624 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:34.834017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:34.834711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.834990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.835263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.836023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:34.836360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:34.837765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:34.852132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 83/168] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:34.853000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.866315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:34.867183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:34.867467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:34.867744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:34.868969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:34.870694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:34.872425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/169] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:34.873193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:34.906680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:35.025956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1e660-0bf5-41d8-b506-5d12629747ba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:35.083815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:35.085171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:35.085639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:35.086095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:35.086639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb01dc8-e953-44ae-a377-aad2e641f493 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:35.087983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:35.090551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:35.094037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/170] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:35.095295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:35.128532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:35.212658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:35.214077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:35.214554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:35.215011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:35.253193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:35.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:35.277889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:35.278660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:35.279241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 84/171] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:35.300072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:35.303484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:35.312472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f236-5edc-48a0-a630-5b53889eaaf1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:35.392937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:35.393921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:35.394392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:35.394861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:35.396421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=f3447cd66131465b871daa6541b4276c) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:35.396980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:35.399556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:35.410085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/172] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:35 2022] GET /identity/v3/role_assignments?user.id=f3447cd66131465b871daa6541b4276c&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:35.411418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:35.635529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e386c58-a3b0-4445-8d74-047748cafade None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:35.700374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:35.700958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:35.701244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:35.701519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:35.735167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ef344-592f-4358-ab85-4d560aa1590d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:35.767071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:35.818807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:35.819924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:35.820484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:35.820970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:35.821913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:35.822426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:35.824012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:35.837226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:35.838841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:35.839060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/173] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:35.844654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:35.879192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:36.034984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 85/174] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:36.041349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:36.075077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57a74f-9cf9-49b6-8f18-46bcfc8c3166 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:36.163062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d678f08-720a-4302-ad26-1068c4e0fedd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:36.188910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:36.189811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.190099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.190382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] PATH_INFO: `/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.196394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc5b62ae82b42d4bdf8ad4af8b91bf3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:36.196574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:36.197670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:36.199696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/175] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:35 2022] GET /identity/v3/users/8cc5b62ae82b42d4bdf8ad4af8b91bf3 => generated 275 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:36.206819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:36.208062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.208738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.209228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:36.209334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] PATH_INFO: `/v3/projects/6f8edb3ba7a5427996b2b75fa2106439` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.210843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] RBAC: Authorizing `identity:get_project(project_id=6f8edb3ba7a5427996b2b75fa2106439)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:36.212563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:36.214467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 86/176] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:36 2022] GET /identity/v3/projects/6f8edb3ba7a5427996b2b75fa2106439 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:36.223006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:36.328404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f885a-ad03-4f5b-b9ef-007ecf0f5a5b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:36.370930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:36.371951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.372464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.372958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.373984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=6f8edb3ba7a5427996b2b75fa2106439, user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:36.374521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:36.376864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:36.383253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 87/177] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:36 2022] GET /identity/v3/role_assignments?user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3&scope.project.id=6f8edb3ba7a5427996b2b75fa2106439&role.id=13dc310ce9ce466088231180b4040e69 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:36.470824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:36.495420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0391da-6951-47a4-8248-24652a214770 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:36.547494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c918ab9-2c0d-4d69-b672-d1fde54ec960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:36.548496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918ab9-2c0d-4d69-b672-d1fde54ec960 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.548945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918ab9-2c0d-4d69-b672-d1fde54ec960 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.549239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c918ab9-2c0d-4d69-b672-d1fde54ec960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.646447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c918ab9-2c0d-4d69-b672-d1fde54ec960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:36.647888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c918ab9-2c0d-4d69-b672-d1fde54ec960 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:36.775684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:36.793225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/178] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:36.800039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:36.803795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:36.932648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff4513-27fe-4611-aaa1-8c1483f4419f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:36.961856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216c484-9647-480b-8cd3-b50c8269621e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.962172 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216c484-9647-480b-8cd3-b50c8269621e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.962426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216c484-9647-480b-8cd3-b50c8269621e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.963956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/179] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:36.969776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.970070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.970350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.982741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:36.983613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:36.983901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:36.984755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:36.985982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:36.988593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:36.991382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 88/180] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:36 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:36.998444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.016383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:37.017614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:37.152400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/181] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:37.152959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:37.176940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.179237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4540eaa-2076-4f11-991d-04f40557ddee None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.234636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.235959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.237537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.238055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.239177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] RBAC: Adding query filter params (role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:37.240462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:37.242224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e34613-a987-45a1-9f1a-c0291c0c0992 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:37.255291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 89/182] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:36 2022] GET /identity/v3/role_assignments?user.id=8cc5b62ae82b42d4bdf8ad4af8b91bf3&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=7805367da7144187ae1dd2e09705a569 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:37.265754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddbac-6ed8-4199-b6fa-ba4f72381553 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.266079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddbac-6ed8-4199-b6fa-ba4f72381553 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.266328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddbac-6ed8-4199-b6fa-ba4f72381553 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.267757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 90/183] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:17:37.273359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.273671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.273930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.303238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:37.306220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:37.330161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1a41a-9ef5-4eb5-bb17-c5fca10916c1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.368943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:37.404820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.406039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.406551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.407047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.429697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:37.438104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:37.438964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:37.439742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:37.442404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/184] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:37.448285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.448693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 91/185] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:37.474966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.553963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:37.562456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd5e6e-e02c-4825-a2b0-ecab195334a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.591021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccf183-5501-4db0-b79e-6e2c024d04a8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.598381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.599048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.599340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.599619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.600420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:37.600737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:37.602125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd8d7-2613-4445-8f0b-932751a0af1d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:37.615751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/186] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:37.615751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61dc04-3f13-4d19-8e92-7f6f2cea0074 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.615751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61dc04-3f13-4d19-8e92-7f6f2cea0074 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.615751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61dc04-3f13-4d19-8e92-7f6f2cea0074 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.615751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/187] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:37.620815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1209e59-e4f5-4c62-bf9b-4bdfc6932bcc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.620815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1209e59-e4f5-4c62-bf9b-4bdfc6932bcc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.621016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1209e59-e4f5-4c62-bf9b-4bdfc6932bcc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.622240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/188] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:37.622985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1209e59-e4f5-4c62-bf9b-4bdfc6932bcc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.634193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237e4ad-a54b-491e-a888-918ca2b3964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.635387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e4ad-a54b-491e-a888-918ca2b3964b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.635679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e4ad-a54b-491e-a888-918ca2b3964b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.636314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e4ad-a54b-491e-a888-918ca2b3964b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.646939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237e4ad-a54b-491e-a888-918ca2b3964b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f237e4ad-a54b-491e-a888-918ca2b3964b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:37.655294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:37.656785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:37.657542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 92/189] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:37.658455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ad0a3-88ae-4cae-9d1b-ba6e39b65a10 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.658760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ad0a3-88ae-4cae-9d1b-ba6e39b65a10 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.659045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ad0a3-88ae-4cae-9d1b-ba6e39b65a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.669368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:37.675308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0e0ad0a3-88ae-4cae-9d1b-ba6e39b65a10 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:37.676522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0e0ad0a3-88ae-4cae-9d1b-ba6e39b65a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:37.727888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1209e59-e4f5-4c62-bf9b-4bdfc6932bcc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.741827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 93/190] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:37.743138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.743430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.743744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.763757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.765219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.765504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.765778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] PATH_INFO: `/v3/users/025f9a8c1c314cfaadbfcbbc019c7136` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.766797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:37.767976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:37.779521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] RBAC: Authorizing `identity:get_user(user_id=025f9a8c1c314cfaadbfcbbc019c7136)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:37.781522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:37.783342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/191] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/users/025f9a8c1c314cfaadbfcbbc019c7136 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:37.783960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.819007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:37.836366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 94/192] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:37.837244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.884335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:37.888595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1e98a-fca7-43e0-8c9c-5e95881b04c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.922212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.922870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.923155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.923440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.924199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:37.924505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:37.925862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:37.933488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/193] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:37.934262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:37.934672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86087b07-c6d1-4a47-8ad5-c4d3ac133b99 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:37.970502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:37.971278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:37.971564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:37.971843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:37.973036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:37.974492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:37.976103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 95/194] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:37.981092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.030028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae2b3d-5efa-4499-aa81-41c41a375592 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.034533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.064722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.065802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.066331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.066851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.080396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a19a-949c-45c3-a2cb-31b6d40f4610 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.094390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.114396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.115180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.115464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.115741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.126986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:38.134667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:38.135525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:38.136334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:38.136911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.136983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 96/195] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:38.136983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.139077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.233865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e46bf-52c8-4a77-991f-f060bac3bbdb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.236594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/196] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:38.237347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.248370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.268967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.269832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.270112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.270606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] PATH_INFO: `/v3/users/025f9a8c1c314cfaadbfcbbc019c7136` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.271556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] RBAC: Authorizing `identity:get_user(user_id=025f9a8c1c314cfaadbfcbbc019c7136)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.273033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.274658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 97/197] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/users/025f9a8c1c314cfaadbfcbbc019c7136 => generated 283 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:38.275411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.299739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.343384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa76fc-251b-4b9a-82ad-9568b099ce16 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.376423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.377086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.377374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.377654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.378363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.379809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.380969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40dba9-f927-40f0-ba1b-b10c54704d9a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.399662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/198] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:38 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:17:38.399662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.431784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.432629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.432915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.433198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.433970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:38.434282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.435712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.445594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 98/199] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:38.446257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.495398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.517670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a737c8-69a0-4e06-b9c1-cc91e2eec798 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.536419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.559054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.559919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.560227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.560512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.563018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.564483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.565944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194da90f-ad21-49e1-8707-3778519de931 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.566398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/200] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:38.566945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.613920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.614588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.614884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.615168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.699889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.701437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.702059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae4ce7-f9d9-4e81-84ec-947766be2db1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.737943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.739751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.739751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.739751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] PATH_INFO: `/v3/users/49a7845a07424dc8a3b526cae02fb738` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.740715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] RBAC: Authorizing `identity:get_user(user_id=49a7845a07424dc8a3b526cae02fb738)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.742299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.744953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/201] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/users/49a7845a07424dc8a3b526cae02fb738 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:38.746068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.760259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.814517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:38.832745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 99/202] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:38.833818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.880721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0bf2d-c123-46a7-afd0-1d87167cfb19 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:38.917402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:38.918598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:38.918598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:38.918837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:38.919534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=025f9a8c1c314cfaadbfcbbc019c7136) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:38.919852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:38.921863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:38.927807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/203] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/role_assignments?user.id=025f9a8c1c314cfaadbfcbbc019c7136&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:38.928638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:38.960787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fc143-301a-4b2a-a1c8-6f0a8b7299c5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:39.004691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:39.005712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:39.006246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:39.006778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:39.025033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:39.100328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:39.102040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:39.117145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749777d6-e1be-4046-8e7d-04a71070951d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:39.145167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:39.185490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:39.186360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:39.186646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:39.186950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:39.192905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:39.194367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:39.196209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/204] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:39.204207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:39.276915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 100/205] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:39.282394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:39.359379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:39.368091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf5b8d-70ed-46c6-b297-a53fae53bdd3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:39.407083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:39.407997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:39.408590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:39.409038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:39.410188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] RBAC: Adding query filter params (role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=025f9a8c1c314cfaadbfcbbc019c7136) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:39.411036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:39.412680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:39.421415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/206] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:39 2022] GET /identity/v3/role_assignments?user.id=025f9a8c1c314cfaadbfcbbc019c7136&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=7805367da7144187ae1dd2e09705a569 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:39.462318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5c2c-4538-474d-a3a8-51d74e27d68a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:39.501448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ac83a-210b-4d8f-86b1-a5585041e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:39.502611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac83a-210b-4d8f-86b1-a5585041e349 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:39.503047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac83a-210b-4d8f-86b1-a5585041e349 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:39.503470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac83a-210b-4d8f-86b1-a5585041e349 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:39.509136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ac83a-210b-4d8f-86b1-a5585041e349 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:39.510917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ac83a-210b-4d8f-86b1-a5585041e349 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:39.513026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 101/207] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:39 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:39.519147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:39.584798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:39.672735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddb9c3-d76d-4f65-a8eb-3c175242662d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:39.730262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:39.730892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:39.731177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:39.731453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:39.732321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] RBAC: Adding query filter params (role.id=f74f6ab86814400fb36f9d8a394c9a90, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d, user.id=49a7845a07424dc8a3b526cae02fb738) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:39.732637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:39.740762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d46271-a0ec-48f2-8ce5-ffd63f669b65 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:39.747529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/208] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:39 2022] GET /identity/v3/role_assignments?user.id=49a7845a07424dc8a3b526cae02fb738&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=f74f6ab86814400fb36f9d8a394c9a90 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:40.836909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece961e5-d601-436f-b116-1c67515e96aa None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:40.837444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece961e5-d601-436f-b116-1c67515e96aa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:40.837444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece961e5-d601-436f-b116-1c67515e96aa None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:40.839119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 102/209] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:40.845300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:40.846302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:40.846736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa657a4-6303-4e14-9d30-821a1a932f81 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:40.847018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:40.847119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa657a4-6303-4e14-9d30-821a1a932f81 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:40.847300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa657a4-6303-4e14-9d30-821a1a932f81 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:40.848918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 103/210] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:40.854065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:40.854432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:40.854792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:40.869330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:40.870779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:40.872580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:40.873810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:40.951965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 104/211] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:40.959952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/212] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:40.977024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:40.989428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.031852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.043952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.108426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8958e-c976-465d-87f9-a4a08a2345c2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.115438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325cf8f-1fa1-4446-b6d3-107ebc7ec3b3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.151738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.152489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.152775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.153054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.154230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:41.155721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:41.157676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/213] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:41.164650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffc31d-d8b2-4dbf-9c9f-637be8027636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.164809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.165854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc31d-d8b2-4dbf-9c9f-637be8027636 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc31d-d8b2-4dbf-9c9f-637be8027636 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.166711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffc31d-d8b2-4dbf-9c9f-637be8027636 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.181695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffc31d-d8b2-4dbf-9c9f-637be8027636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7bffc31d-d8b2-4dbf-9c9f-637be8027636 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:41.192570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:41.193593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:41.194543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:41.196569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 105/214] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:41.198048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629aa39-d90b-4bcb-9d18-78eb85177a65 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.198500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629aa39-d90b-4bcb-9d18-78eb85177a65 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.198926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629aa39-d90b-4bcb-9d18-78eb85177a65 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.200392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 106/215] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:41.202182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629aa39-d90b-4bcb-9d18-78eb85177a65 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.279863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f835936-60ad-43f7-8c54-070a7354745b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.294149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.319121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629aa39-d90b-4bcb-9d18-78eb85177a65 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.320146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602a573-96a4-407b-bd55-67dde4e0b9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.320750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.320997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a573-96a4-407b-bd55-67dde4e0b9dd None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.321288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a573-96a4-407b-bd55-67dde4e0b9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.322371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602a573-96a4-407b-bd55-67dde4e0b9dd None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.323103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602a573-96a4-407b-bd55-67dde4e0b9dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:41.324589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602a573-96a4-407b-bd55-67dde4e0b9dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:41.347824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/216] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:41 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:41.357822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.358239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.358630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.359545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.360503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.361099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.361655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.362748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:41.363360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:41.365384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:41.378309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 107/217] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:41.384255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:41.384255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:41.385581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.467675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/218] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:41.490660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.517928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372be0-c16e-4e7a-891e-67f15894a4cb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.565733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.566529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.567092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.567382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] PATH_INFO: `/v3/users/10347de7c65c46559672a0b2c9951414` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.587253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] RBAC: Authorizing `identity:get_user(user_id=10347de7c65c46559672a0b2c9951414)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:41.588860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:41.595789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 108/219] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:41 2022] GET /identity/v3/users/10347de7c65c46559672a0b2c9951414 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:41.596689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.612292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e9668-18e3-48b4-9b74-c697415ae56c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.623406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.629814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.662476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.663644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.664016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.664482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.679457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:41.689629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:41.690525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:41.691280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/220] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:41.697812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.729513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7a344-e349-4ee4-ae52-e22f0afd77ad None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.767998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94ceba-2dc4-438a-b15d-006b6608ff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.768690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94ceba-2dc4-438a-b15d-006b6608ff18 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.768973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94ceba-2dc4-438a-b15d-006b6608ff18 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.769253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94ceba-2dc4-438a-b15d-006b6608ff18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.826010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32718-0db3-4b2e-b325-14a7da162b69 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:41.846683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94ceba-2dc4-438a-b15d-006b6608ff18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:41.848376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94ceba-2dc4-438a-b15d-006b6608ff18 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:41.868389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:41.869312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.869599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.869878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.871060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:41.871370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:41.873066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e0804-e4ac-44b5-83a2-75efd91668d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:41.873174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.883475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/221] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:41.884870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fd3df-8372-4b5b-af1b-0205911c57c4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.885158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fd3df-8372-4b5b-af1b-0205911c57c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.885437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fd3df-8372-4b5b-af1b-0205911c57c4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.886673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/222] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:41.891790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fd3df-8372-4b5b-af1b-0205911c57c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:41.912052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:41.961029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 109/223] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:41.962630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:41.962919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:41.963193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:41.980331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:41.981555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:41.998133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fd3df-8372-4b5b-af1b-0205911c57c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.032711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.033581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.033868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.034143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] PATH_INFO: `/v3/users/025f9a8c1c314cfaadbfcbbc019c7136` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.035361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] RBAC: Authorizing `identity:get_user(user_id=025f9a8c1c314cfaadbfcbbc019c7136)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.036852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.038655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/224] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:41 2022] GET /identity/v3/users/025f9a8c1c314cfaadbfcbbc019c7136 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:42.039249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.060276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 110/225] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:42.061170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.138050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:42.161700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:42.181343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7da6a-8db4-4f9c-9a81-1af64022efaa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.197183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9669d3-ac4d-4df1-9808-5428108987cd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.231522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.232635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.233087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.233503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.234006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.234870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.235152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.235359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.235464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.236642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.237345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.238103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.239594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/226] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:42.239782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 111/227] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:42.240688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.249211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.350635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59bd3-aa09-461c-8abe-644f34a8398a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.357470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd604b-1aa5-43b9-a3eb-1fd588aefac4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.379349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:42.388258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.389367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.389949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.390441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.394923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.395742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.396145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.396468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.397311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=10347de7c65c46559672a0b2c9951414) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:42.397633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.399383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.404874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:42.411197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 112/228] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/role_assignments?user.id=10347de7c65c46559672a0b2c9951414&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=13dc310ce9ce466088231180b4040e69 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:42.412356 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:42.413598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:42.414456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:42.415620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/229] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:42.421059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.439019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:42.531316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c5f59-78e7-4efa-9202-e2ca314bfd13 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.554884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96362c1b-5542-44a1-9913-21b25444a91d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.568376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.569083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.569489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.569869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.570773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:42.571180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.572746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.581464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/230] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:42.587336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.616705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.618845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.618845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.619455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f/users/025f9a8c1c314cfaadbfcbbc019c7136/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.624945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] RBAC: Authorizing `identity:create_grant(project_id=3fc57ef585554a139d14be6c444c334f, user_id=025f9a8c1c314cfaadbfcbbc019c7136, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.626601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.654627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 113/231] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:42 2022] PUT /identity/v3/projects/3fc57ef585554a139d14be6c444c334f/users/025f9a8c1c314cfaadbfcbbc019c7136/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:42.712707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:42.734737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be827-f60f-4c70-b236-88092bb48577 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.776247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.777238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.777527 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.777803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] PATH_INFO: `/v3/users/025f9a8c1c314cfaadbfcbbc019c7136` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.779092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=025f9a8c1c314cfaadbfcbbc019c7136)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.780751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.782778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/232] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/users/025f9a8c1c314cfaadbfcbbc019c7136 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:42.788289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:42.841595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:42.901711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066e1d0-938e-4735-8b9f-6f9cc542dabd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:42.936318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3264b5f-5c4d-44ad-ade4-a85c5e837d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:42.937288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264b5f-5c4d-44ad-ade4-a85c5e837d2e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:42.937572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264b5f-5c4d-44ad-ade4-a85c5e837d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:42.937847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3264b5f-5c4d-44ad-ade4-a85c5e837d2e None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:42.939050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3264b5f-5c4d-44ad-ade4-a85c5e837d2e None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:42.940533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3264b5f-5c4d-44ad-ade4-a85c5e837d2e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:42.942229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 114/233] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:42 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:42.947974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:43.061050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb98a81-2f1a-4e5e-a183-ebf6f2fa8cd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:43.111704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ec8e1-28c9-43fb-a290-1f693cd37514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:43.112620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ec8e1-28c9-43fb-a290-1f693cd37514 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:43.112902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ec8e1-28c9-43fb-a290-1f693cd37514 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:43.113176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ec8e1-28c9-43fb-a290-1f693cd37514 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f/users/025f9a8c1c314cfaadbfcbbc019c7136/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:43.121509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ec8e1-28c9-43fb-a290-1f693cd37514 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fc57ef585554a139d14be6c444c334f, user_id=025f9a8c1c314cfaadbfcbbc019c7136, role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:43.122969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ec8e1-28c9-43fb-a290-1f693cd37514 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:43.147233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/234] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:42 2022] PUT /identity/v3/projects/3fc57ef585554a139d14be6c444c334f/users/025f9a8c1c314cfaadbfcbbc019c7136/roles/7805367da7144187ae1dd2e09705a569 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:43.808689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e775ae4-889f-4826-b6ed-d99584145eb4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:43.809084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e775ae4-889f-4826-b6ed-d99584145eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:43.809264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e775ae4-889f-4826-b6ed-d99584145eb4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:43.810865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 115/235] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:43.816690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:43.816984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:43.817267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:43.826537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:43.839198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:43.840454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:43.909654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31309e65-bcea-485b-ab06-8bea1395f73b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:43.910130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31309e65-bcea-485b-ab06-8bea1395f73b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:43.910130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31309e65-bcea-485b-ab06-8bea1395f73b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:43.911607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 116/236] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:43.917942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:43.918717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:43.919295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:43.939210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:43.940016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/237] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:43.940969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:43.965613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:44.066500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 117/238] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:44.085402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d953f-f3d5-46f3-8634-9af8929f5b75 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:44.101785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:44.125188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:44.125994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.126280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.126559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:44.131428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:44.132912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:44.137831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/239] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:44.143803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:44.146851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:44.187735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:44.293303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113a4a4-4d1d-4f50-98a3-4843b7cdbd1d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:44.321431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e8cb6-1593-4282-bdf4-b5bd9125d320 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:44.336705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:44.337397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.337692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.337976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:44.338720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:44.340258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:44.382647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/240] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:44 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:44.385372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:44.386575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.387084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.387604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:44.403149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:44.416514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:44.418010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:44.419348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:44.423285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 118/241] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:44.428894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:44.498926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:44.548443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159600-2f42-4024-a0e1-fcab62e5d6fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:44.586191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:44.587247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.587761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.588272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:44.589354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:44.589904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:44.591598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:44.602687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/242] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:44.611155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:44.748725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b6ee3-2944-4413-89f0-138d4b25dbc9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:44.750878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:44.788325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a40c35-a288-4508-8914-556b27848411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:44.789207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a40c35-a288-4508-8914-556b27848411 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.789497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a40c35-a288-4508-8914-556b27848411 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.789779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a40c35-a288-4508-8914-556b27848411 None admin] PATH_INFO: `/v3/users/93ef45bee69246539851c55b03510ddd` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:44.803431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a40c35-a288-4508-8914-556b27848411 None admin] RBAC: Authorizing `identity:get_user(user_id=93ef45bee69246539851c55b03510ddd)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:44.804954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a40c35-a288-4508-8914-556b27848411 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:44.806640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 119/243] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:44 2022] GET /identity/v3/users/93ef45bee69246539851c55b03510ddd => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:44.812994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:44.919137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc8f40-1717-4fef-a993-3fbcd59b71b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:44.941017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:44.958567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:44.959235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.959577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.959855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:44.999288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c030d-c7ed-44dd-bd56-95ab79621b54 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:44.999789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c030d-c7ed-44dd-bd56-95ab79621b54 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:44.999860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c030d-c7ed-44dd-bd56-95ab79621b54 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.005437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 120/244] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:45.016136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.016773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.017640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.040061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:45.040380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:45.041845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:45.042378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:45.143559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 121/245] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:45.160012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/246] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:45.165882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:45.169274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:45.195044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:45.304672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:45.305706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd26f7-512a-43b1-8670-fe079699462a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:45.320832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41383c8a-c0fe-47af-b803-45c284f34ef6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:45.379253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:45.380132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.380447 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.380787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.382040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:45.383567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:45.385655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 122/247] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:45 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:45.387874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:45.388732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.389016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.389292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.393219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:45.403359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:45.411817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:45.412732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:45.413504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/248] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:45.420608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:45.517930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51a7a-45d6-4224-896b-c2bd896b8552 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:45.556328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:45.557000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.557292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.557574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.558419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] RBAC: Adding query filter params (role.id=f74f6ab86814400fb36f9d8a394c9a90, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=93ef45bee69246539851c55b03510ddd) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:45.558734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:45.560359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6b9f5-1c0d-41dd-8174-4f81195c4989 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:45.566791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 123/249] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:45 2022] GET /identity/v3/role_assignments?user.id=93ef45bee69246539851c55b03510ddd&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=f74f6ab86814400fb36f9d8a394c9a90 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:45.614354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:45.615894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c11dd-465f-4bed-b34f-6ae7ec4f3254 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.616486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c11dd-465f-4bed-b34f-6ae7ec4f3254 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.616998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c11dd-465f-4bed-b34f-6ae7ec4f3254 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.618707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 124/250] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:45.626409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.627010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.627494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.647163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:45.648783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:45.670667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe0c3d-ecfe-4416-a131-c0c6bd51bfb1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:45.702007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:45.761301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 125/251] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:45.789931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:45.790618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.790913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.791177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.791949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:45.792281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:45.793689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823a11f-cda3-45c8-9d6e-d44fc4b312b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:45.796502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:45.809542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/252] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:45.810809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a7f0e-c657-4bbe-9f2f-43a59da971fc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:45.811120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a7f0e-c657-4bbe-9f2f-43a59da971fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:45.811379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a7f0e-c657-4bbe-9f2f-43a59da971fc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:45.813057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/253] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:45.817325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a7f0e-c657-4bbe-9f2f-43a59da971fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:45.966705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463c5a9-9eac-441c-a4ed-53f05c16cb40 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:45.981726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a7f0e-c657-4bbe-9f2f-43a59da971fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.042154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.042966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.043248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.043527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.054654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f011787-09ba-454b-873f-c8d6f909241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.055550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011787-09ba-454b-873f-c8d6f909241a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.055838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011787-09ba-454b-873f-c8d6f909241a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.056116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f011787-09ba-454b-873f-c8d6f909241a None admin] PATH_INFO: `/v3/users/10347de7c65c46559672a0b2c9951414` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.059844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f011787-09ba-454b-873f-c8d6f909241a None admin] RBAC: Authorizing `identity:get_user(user_id=10347de7c65c46559672a0b2c9951414)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:46.061304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f011787-09ba-454b-873f-c8d6f909241a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:46.067443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/254] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:45 2022] GET /identity/v3/users/10347de7c65c46559672a0b2c9951414 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:46.068794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.069087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.069369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.070970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:46.092532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:46.094007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:46.094771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 126/255] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:46.095896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.098347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:46.108391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:46.120514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:46.136532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:46.245329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/256] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:46.246532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.252302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423ca4-5bfb-4beb-a0db-3592157d43b7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.294613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.295839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.296165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.296474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.297937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:46.299837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:46.301579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 127/257] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:46.302419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.364446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d9eb3-5071-4c6f-b989-4e59f0eb8387 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.393044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:46.409018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.409707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.409994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.410273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.411311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:46.411624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:46.413549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:46.416115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73aeb9-2bd5-45cc-803d-aedbe3575fe0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.423720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/258] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:46.425069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.427963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:46.454098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.455285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.455774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.456321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.467207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:46.477573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:46.478438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:46.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:46.481588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 128/259] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:46.482668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.550590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1347ed2-0676-4274-be35-9f23d836f2ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.585955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.586857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.587145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.587501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f/users/10347de7c65c46559672a0b2c9951414/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.589582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] RBAC: Authorizing `identity:create_grant(project_id=3fc57ef585554a139d14be6c444c334f, user_id=10347de7c65c46559672a0b2c9951414, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:46.591093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:46.593964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825c594-4d4a-465b-bae7-fe6c684035e5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.633821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/260] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:46 2022] PUT /identity/v3/projects/3fc57ef585554a139d14be6c444c334f/users/10347de7c65c46559672a0b2c9951414/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:46.635658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.638405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.641106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.642537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.642892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] PATH_INFO: `/v3/users/025f9a8c1c314cfaadbfcbbc019c7136` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.645217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] RBAC: Authorizing `identity:get_user(user_id=025f9a8c1c314cfaadbfcbbc019c7136)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:46.646983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:46.653014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 129/261] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/users/025f9a8c1c314cfaadbfcbbc019c7136 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:46.658835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.679292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:46.728579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:46.816728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8eaec-0bea-4618-9e7f-f98e8a2c678f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:46.876327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:46.878521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:46.879028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:46.879520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:46.880562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:46.881082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:46.882696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:46.893274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/262] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:46.900685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:46.942247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3428fe9-688e-4303-896f-60343225a815 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:47.002520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:47.003174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.003457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.003735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.090034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:47.092401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:47.108919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:47.151212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745818a-668d-458b-a7d4-bc302e700cce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:47.195798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:47.206633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 130/263] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:17:47.212688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:47.229438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:47.230372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.230598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.230900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] PATH_INFO: `/v3/users/025f9a8c1c314cfaadbfcbbc019c7136` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.236923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] RBAC: Authorizing `identity:get_user(user_id=025f9a8c1c314cfaadbfcbbc019c7136)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:47.238399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:47.240418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/264] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:46 2022] GET /identity/v3/users/025f9a8c1c314cfaadbfcbbc019c7136 => generated 283 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:47.245646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:47.334243 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc7225-8249-4aea-abbf-cb3e247cc62f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:47.346565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:47.379280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:47.380846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.381341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.381839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.383304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:47.385844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:47.388967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 131/265] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:47 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:47.397279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:47.482811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bae4e9-0fff-442e-a574-bdafa7637741 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:47.542876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa03e32-065f-4f85-a318-d25cbf8bf234 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:47.560915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f21ff6-e1aa-4e5a-9b76-4b48bd420f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:47.561616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f21ff6-e1aa-4e5a-9b76-4b48bd420f72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.561899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f21ff6-e1aa-4e5a-9b76-4b48bd420f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.562172 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f21ff6-e1aa-4e5a-9b76-4b48bd420f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.628157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:47.628845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.629126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.629403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.630247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=025f9a8c1c314cfaadbfcbbc019c7136) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:47.630557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:47.632230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:47.650617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 132/266] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:47 2022] GET /identity/v3/role_assignments?user.id=025f9a8c1c314cfaadbfcbbc019c7136&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:47.677078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:47.690174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f21ff6-e1aa-4e5a-9b76-4b48bd420f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:47.692009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f21ff6-e1aa-4e5a-9b76-4b48bd420f72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:47.804762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/267] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:17:47.813298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:47.825166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6fb47-340b-435b-9f23-4fb1d74894e0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.828883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6fb47-340b-435b-9f23-4fb1d74894e0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.829480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6fb47-340b-435b-9f23-4fb1d74894e0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.837313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/268] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:47 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:47.843833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19047a34-b587-489f-8204-1be8930bde91 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:47.844468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19047a34-b587-489f-8204-1be8930bde91 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:47.844954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19047a34-b587-489f-8204-1be8930bde91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:47.894433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-19047a34-b587-489f-8204-1be8930bde91 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:47.897711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19047a34-b587-489f-8204-1be8930bde91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:47.912037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.018911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/269] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:48.020964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5defd-7781-40d7-85df-a725e2b309e6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.021757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5defd-7781-40d7-85df-a725e2b309e6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.022253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5defd-7781-40d7-85df-a725e2b309e6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.023436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf0a10-7382-409d-ad06-bf1056e28e19 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.024009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/270] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:48.033592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d7c18-257f-410a-b43a-284d31765155 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.034180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d7c18-257f-410a-b43a-284d31765155 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.035089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d7c18-257f-410a-b43a-284d31765155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.063231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-317d7c18-257f-410a-b43a-284d31765155 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:48.065073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-317d7c18-257f-410a-b43a-284d31765155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:48.087183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.088248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.089681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.090066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.091380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:48.094307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:48.095217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 133/271] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:47 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:48.096499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.164356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/272] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:48.165758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d7c18-257f-410a-b43a-284d31765155 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.236539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c18b07-39c1-4888-9b5e-c5be31849a6b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.246329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.269286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d7c18-257f-410a-b43a-284d31765155 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.274062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.274943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.275320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.275673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.287298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:48.295323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:48.296296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:48.297086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 134/273] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:48.302198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.309926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.310868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.311356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.312439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.313550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] RBAC: Adding query filter params (role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=025f9a8c1c314cfaadbfcbbc019c7136) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:48.313875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:48.315398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:48.321751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/274] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/role_assignments?user.id=025f9a8c1c314cfaadbfcbbc019c7136&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=7805367da7144187ae1dd2e09705a569 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:48.322679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.342275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.424063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ba316-3916-4053-9e8e-8c47ae59226b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.464440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.465703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.466175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.466661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.468845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:48.470617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:48.472579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 135/275] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:48.478507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.542810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.567608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dc089-a880-4b38-a254-91a4aabe458c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.600613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb24c3-1d14-4158-be71-c1d632458961 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.613223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.615376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.615376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.615376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.615811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:48.617506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:48.618192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd63efa-64ce-49f8-b650-7c656b877196 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:48.629247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/276] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:48.630828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f47c2-0048-429e-9d60-dcd1c587c0d6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.631106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f47c2-0048-429e-9d60-dcd1c587c0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.631404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f47c2-0048-429e-9d60-dcd1c587c0d6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.633182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/277] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:48.637863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f47c2-0048-429e-9d60-dcd1c587c0d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.643959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f4e4f-39eb-46f0-84d8-22af26ec4086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.644730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f4e4f-39eb-46f0-84d8-22af26ec4086 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.645013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f4e4f-39eb-46f0-84d8-22af26ec4086 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.645372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f4e4f-39eb-46f0-84d8-22af26ec4086 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.646095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f4e4f-39eb-46f0-84d8-22af26ec4086 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:48.647557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f4e4f-39eb-46f0-84d8-22af26ec4086 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:48.670459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 136/278] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:17:48 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:48.672271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.672556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.672851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.697593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:48.698816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:48.752090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.786605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f47c2-0048-429e-9d60-dcd1c587c0d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.801600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 137/279] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:48.825268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.832894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.833793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.834084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.834366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] PATH_INFO: `/v3/users/e34199722ba648358b9bc2fe2ba7e0cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:48.850851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] RBAC: Authorizing `identity:get_user(user_id=e34199722ba648358b9bc2fe2ba7e0cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:48.852396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:48.854141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/280] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/users/e34199722ba648358b9bc2fe2ba7e0cb => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:48.855231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:48.860512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:48.947566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d9823-f925-43e2-ab75-1c59c7d961a4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.981986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdaf5ad-a6ed-468f-bbb1-0b982b17ec10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:48.987613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:48.988453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:48.988738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:48.989015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.005298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:49.013631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:49.014504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:49.015919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 138/281] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:49.021674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:49.035256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:49.036856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.037217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.037876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.039502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, user.id=e34199722ba648358b9bc2fe2ba7e0cb, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:49.039502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:49.041183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:49.044797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:49.049886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/282] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jul 19 12:17:48 2022] GET /identity/v3/role_assignments?user.id=e34199722ba648358b9bc2fe2ba7e0cb&scope.system=all&role.id=13dc310ce9ce466088231180b4040e69 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:49.125828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:49.152475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38584987-5643-4186-af1d-9d6f2d15cd60 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:49.196409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:49.197467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.197948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.198436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.199490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:49.200004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:49.201628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35bb28-018a-4bfd-8d2b-da55a62a9aa9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:49.211382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 139/283] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:49.217475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:49.333815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258010cd-95e0-4282-98ed-0ca15fa73ba5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:49.375947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548055d0-3a6b-46a4-904a-28bb7a13122a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.376269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548055d0-3a6b-46a4-904a-28bb7a13122a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.376468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548055d0-3a6b-46a4-904a-28bb7a13122a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.376852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:49.377732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.378057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 140/284] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:49.378159 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.378394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] PATH_INFO: `/v3/users/93ef45bee69246539851c55b03510ddd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.379698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=93ef45bee69246539851c55b03510ddd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:49.381194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:49.382928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/285] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:49 2022] GET /identity/v3/users/93ef45bee69246539851c55b03510ddd => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:49.384317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.384611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.384895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.389269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:49.418423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:49.419637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:49.468779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:49.512213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e34b21-138b-420e-ac65-6251d7fac3d6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:49.554336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 141/286] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:49.555072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:49.556240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.556726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.557199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.558616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:49.560409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:49.562369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/287] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:49 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:49.568165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:49.585891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:49.725554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:49.747336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf26e5-8b4f-463a-884e-426109375772 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:49.754897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b8e2c-3e64-4546-b76c-ae3a3e157c6a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:49.823477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:49.824304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.824599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.824883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.832859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:49.834130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:49.835075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:49.835324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:49.835580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f/users/93ef45bee69246539851c55b03510ddd/roles/f74f6ab86814400fb36f9d8a394c9a90` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:49.839015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fc57ef585554a139d14be6c444c334f, user_id=93ef45bee69246539851c55b03510ddd, role_id=f74f6ab86814400fb36f9d8a394c9a90)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:49.840631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:49.851304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:49.857769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 142/288] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:17:49 2022] PUT /identity/v3/projects/3fc57ef585554a139d14be6c444c334f/users/93ef45bee69246539851c55b03510ddd/roles/f74f6ab86814400fb36f9d8a394c9a90 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:49.859436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:49.860431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:49.861213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:49.865222 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/289] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:49.873415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:50.048295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45834e-f817-4a9b-a652-e56c254d6472 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:50.089607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:50.090274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:50.090561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:50.090841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:50.091596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:50.091904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:50.095925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:50.098086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:50.109679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 143/290] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:50.116318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:50.243365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fe9e-4b54-42eb-b7d7-25ea0aad0715 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:50.283909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:50.284879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:50.285164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:50.285439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] PATH_INFO: `/v3/users/10347de7c65c46559672a0b2c9951414` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:50.289212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] RBAC: Authorizing `identity:get_user(user_id=10347de7c65c46559672a0b2c9951414)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:50.290704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:50.292727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/291] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:50 2022] GET /identity/v3/users/10347de7c65c46559672a0b2c9951414 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:50.298406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:50.461117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d1014-e482-4302-9ecb-7a4b2f772c1b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:50.511964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:50.512954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:50.513375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:50.513791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:50.614340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:50.619879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:50.621666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:50.759662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 144/292] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:50.767134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:50.867483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:50.914361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6ec8a-1df9-4452-bcc6-4ec454aa47ff None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:50.963272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78830b-0867-4f53-8c57-e8612ce06dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:50.964145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78830b-0867-4f53-8c57-e8612ce06dec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:50.964493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78830b-0867-4f53-8c57-e8612ce06dec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:50.964777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78830b-0867-4f53-8c57-e8612ce06dec None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:50.965990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78830b-0867-4f53-8c57-e8612ce06dec None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:50.967419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c78830b-0867-4f53-8c57-e8612ce06dec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:50.969501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/293] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:50 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:50.975010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:51.126717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:51.132853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae7924-b127-4ec3-9230-940f27e3c79c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:51.173206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:51.173871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.174159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.174443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.175280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=10347de7c65c46559672a0b2c9951414) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:51.175597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:51.177091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd189f-e124-40fe-a47e-b38fa4dbc0dc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:51.183058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 145/294] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:50 2022] GET /identity/v3/role_assignments?user.id=10347de7c65c46559672a0b2c9951414&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=13dc310ce9ce466088231180b4040e69 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:51.423111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffd832-38cf-48fb-8306-b7ec583eec2e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.423587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffd832-38cf-48fb-8306-b7ec583eec2e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.423587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffd832-38cf-48fb-8306-b7ec583eec2e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.424971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/295] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:51.430662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.431088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.431264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.450650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:51.451845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:51.525843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 146/296] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:51.542109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3b67-fd01-4cfc-b24c-83d5d78eb2e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.542396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3b67-fd01-4cfc-b24c-83d5d78eb2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.542674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b3b67-fd01-4cfc-b24c-83d5d78eb2e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.544058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/297] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:51.548393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:51.552263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.552579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.552848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.573331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:51.574695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:51.662307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/298] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:51.679497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:51.689465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:51.698747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4b392-ff1d-45ee-a4ae-b0c21259015f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:51.735162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:51.735955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.736264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.736543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.737693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:51.739148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:51.740995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 147/299] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:51.746384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:51.757230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:51.801463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a51669-ab2e-40f1-a219-0fe75a696560 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:51.838502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bca5f-6448-4d47-a195-5eff4f0469f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:51.839311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bca5f-6448-4d47-a195-5eff4f0469f3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.839596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bca5f-6448-4d47-a195-5eff4f0469f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.839872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bca5f-6448-4d47-a195-5eff4f0469f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.852007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bca5f-6448-4d47-a195-5eff4f0469f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:51.855307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfab5a-41d8-4692-aafa-3620ae6c96c1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-832bca5f-6448-4d47-a195-5eff4f0469f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:51.859764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:51.860676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:51.861877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/300] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:51.866949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71acbb-5743-45a5-88ab-0e24c8e3e5d8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.867243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71acbb-5743-45a5-88ab-0e24c8e3e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.867529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a71acbb-5743-45a5-88ab-0e24c8e3e5d8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.872218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/301] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:51.872218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71acbb-5743-45a5-88ab-0e24c8e3e5d8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:51.896602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730dab03-0e5a-44b8-a061-6947973bb30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:51.897587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dab03-0e5a-44b8-a061-6947973bb30e None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.897875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dab03-0e5a-44b8-a061-6947973bb30e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.898296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730dab03-0e5a-44b8-a061-6947973bb30e None admin] PATH_INFO: `/v3/groups` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.898983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dab03-0e5a-44b8-a061-6947973bb30e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:51.900549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730dab03-0e5a-44b8-a061-6947973bb30e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:51.915439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 148/302] 10.222.0.13 () {60 vars in 1134 bytes} [Tue Jul 19 12:17:51 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:51.923459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:51.924092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:51.924664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:51.951416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:51.953594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:51.975959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:51.997360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a71acbb-5743-45a5-88ab-0e24c8e3e5d8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.033226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:52.047446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.048131 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.048470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.048756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.049532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:52.056203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.056203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.056897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 149/303] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:52.065845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/304] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:52.071510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.077896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.195546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03c52b-ebbe-4a86-936b-b949628f2db0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.200944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e25bf-ce6c-431b-b76f-7201827800d2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.238490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.240017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.240702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.241279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.242929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.244121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.244501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.244843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] PATH_INFO: `/v3/users/7a6c172094a945e1bd6d6eb519bfb6a5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.255671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:52.261637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] RBAC: Authorizing `identity:get_user(user_id=7a6c172094a945e1bd6d6eb519bfb6a5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.263309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:52.263693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:52.264738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:52.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/305] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:52.271519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 150/306] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/users/7a6c172094a945e1bd6d6eb519bfb6a5 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:52.272790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.277638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.288702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:52.324953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:52.393810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7b66-d67a-4f01-959f-a463663b9b64 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.405647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e6009-a482-4dd0-a61e-97eba335db76 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.431515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.432778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.433071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.433358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.434115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:52.434425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.435801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c76e-692b-424c-bbb5-6e5686db79c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.444455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/307] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:52.444455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb344d36-ce20-411a-b6be-9e350ca67baf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.444455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb344d36-ce20-411a-b6be-9e350ca67baf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.444455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb344d36-ce20-411a-b6be-9e350ca67baf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.444455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/308] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:52.450652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb344d36-ce20-411a-b6be-9e350ca67baf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.454125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.454827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.455070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.455344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.456199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] RBAC: Adding query filter params (role.id=f74f6ab86814400fb36f9d8a394c9a90, user.id=7a6c172094a945e1bd6d6eb519bfb6a5, scope.system=all) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:52.456527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.457973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09151481-18c5-4aa3-b9ac-75de2cb6bc20 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.463589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 151/309] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/role_assignments?user.id=7a6c172094a945e1bd6d6eb519bfb6a5&scope.system=all&role.id=f74f6ab86814400fb36f9d8a394c9a90 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:52.468248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.468248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.468248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.482769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:52.484634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:52.561527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:52.569577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 152/310] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:52.583032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb344d36-ce20-411a-b6be-9e350ca67baf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.593798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:52.597029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.630120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.631013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.631301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.631581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] PATH_INFO: `/v3/users/e34199722ba648358b9bc2fe2ba7e0cb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.633348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] RBAC: Authorizing `identity:get_user(user_id=e34199722ba648358b9bc2fe2ba7e0cb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.634637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.637183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/311] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/users/e34199722ba648358b9bc2fe2ba7e0cb => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:52.642614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.742070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9ef30-7a51-4b2a-a489-e4e2a99dda26 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.755858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385bd02-b4ce-411f-9b4f-dd80a4694924 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.780553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.781355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.781641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.781919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.792618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:52.801096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:52.801378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:52.802197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:52.802989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 153/312] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:52.804984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.805466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.805931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] PATH_INFO: `/v3/system/users/e34199722ba648358b9bc2fe2ba7e0cb/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.808024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e34199722ba648358b9bc2fe2ba7e0cb, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.808577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:52.809931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.828245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/313] 10.222.0.13 () {58 vars in 1334 bytes} [Tue Jul 19 12:17:52 2022] PUT /identity/v3/system/users/e34199722ba648358b9bc2fe2ba7e0cb/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:17:52.897323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:52.940424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee5806-3e82-4ef4-b8c7-b6d7b9adaf56 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:52.978538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:52.979570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:52.980102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:52.980601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:52.981645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:52.982142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:52.983841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:52.993149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 154/314] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:52.999777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:53.045311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:53.111402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366814-1434-4066-b32a-79187bf7bb05 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:53.148561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:53.149436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:53.149721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:53.150001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] PATH_INFO: `/v3/users/93ef45bee69246539851c55b03510ddd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:53.151270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] RBAC: Authorizing `identity:get_user(user_id=93ef45bee69246539851c55b03510ddd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:53.152816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:53.154814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/315] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:52 2022] GET /identity/v3/users/93ef45bee69246539851c55b03510ddd => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:53.162192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:53.271565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f75d4-5c3b-4b3a-88c8-f3dccc08dcdb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:53.295883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:53.309177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:53.310094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:53.310582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:53.311104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:53.385307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:53.387096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:53.493532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 155/316] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:53.501273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:53.616274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c09bca-1225-43b0-a1c4-c7c1bd7c09bc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:53.628564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:53.655839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7984542-997a-4df2-8154-b7679bbea472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:53.656652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7984542-997a-4df2-8154-b7679bbea472 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:53.656932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7984542-997a-4df2-8154-b7679bbea472 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:53.657205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7984542-997a-4df2-8154-b7679bbea472 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:53.658403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7984542-997a-4df2-8154-b7679bbea472 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:53.660283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7984542-997a-4df2-8154-b7679bbea472 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:53.662669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/317] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:53 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:53.669321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:53.695635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:53.781011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968aa2a-6fcb-4f52-bb0b-d40e44115b8b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:53.817039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:53.817704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:53.817988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:53.818264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:53.819096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] RBAC: Adding query filter params (role.id=f74f6ab86814400fb36f9d8a394c9a90, scope.project.id=3fc57ef585554a139d14be6c444c334f, user.id=93ef45bee69246539851c55b03510ddd) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:53.819404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:53.820935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc15fa0-a282-415d-8a73-2a59e9fa2a51 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:53.828027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 156/318] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:17:53 2022] GET /identity/v3/role_assignments?user.id=93ef45bee69246539851c55b03510ddd&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=f74f6ab86814400fb36f9d8a394c9a90 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:54.065559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25d95-a2ab-4ef6-aadf-13aacb37e335 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.065921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25d95-a2ab-4ef6-aadf-13aacb37e335 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.066145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25d95-a2ab-4ef6-aadf-13aacb37e335 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.067701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/319] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:54.072706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.073004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.073288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.091357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:54.093208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:54.164213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 157/320] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:54.186667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e25d95-a2ab-4ef6-aadf-13aacb37e335 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:54.291798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e25d95-a2ab-4ef6-aadf-13aacb37e335 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:54.325296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6969b30-2b3d-4a61-a0ee-86769389717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:54.326011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6969b30-2b3d-4a61-a0ee-86769389717a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.326297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6969b30-2b3d-4a61-a0ee-86769389717a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.326574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6969b30-2b3d-4a61-a0ee-86769389717a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.328340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6969b30-2b3d-4a61-a0ee-86769389717a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:54.329809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6969b30-2b3d-4a61-a0ee-86769389717a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:54.335803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/321] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:17:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:54.337030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:54.354266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:54.436042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d776a88-f234-4e90-967d-f4311de930c8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:54.469594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a5a5c-0b24-4693-8928-861978186466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:54.470260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a5a5c-0b24-4693-8928-861978186466 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.470542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a5a5c-0b24-4693-8928-861978186466 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.470817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a5a5c-0b24-4693-8928-861978186466 None admin] PATH_INFO: `/v3/groups` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.471507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a5a5c-0b24-4693-8928-861978186466 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:54.472989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a5a5c-0b24-4693-8928-861978186466 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:54.487493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 158/322] 10.222.0.13 () {60 vars in 1134 bytes} [Tue Jul 19 12:17:54 2022] POST /identity/v3/groups => generated 226 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:17:54.580767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa258ad-a7fc-4f36-a381-5c31d9d0d5c0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.581279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa258ad-a7fc-4f36-a381-5c31d9d0d5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.581279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa258ad-a7fc-4f36-a381-5c31d9d0d5c0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.582893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/323] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:54.590435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.590925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.591305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.614910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:54.616121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:54.686080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 159/324] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:54.708431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa258ad-a7fc-4f36-a381-5c31d9d0d5c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:54.757486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:54.819549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa258ad-a7fc-4f36-a381-5c31d9d0d5c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:54.857088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f746c-771e-4019-818c-22f8313f57da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:54.858710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f746c-771e-4019-818c-22f8313f57da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:54.859170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f746c-771e-4019-818c-22f8313f57da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:54.859616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f746c-771e-4019-818c-22f8313f57da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:54.870898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f746c-771e-4019-818c-22f8313f57da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-506f746c-771e-4019-818c-22f8313f57da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:54.879161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:54.880034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:17:54.880857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:54.881798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/325] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:54.891271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:54.907048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:55.008205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d095c67-e4d2-4cd1-9865-85c396798ac4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:55.047131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:55.048154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.048672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.049168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.050121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:55.050638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:55.052237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:55.054761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187adbdf-52ce-48d3-bd03-6e59a5005fdc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.055056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187adbdf-52ce-48d3-bd03-6e59a5005fdc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.055353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187adbdf-52ce-48d3-bd03-6e59a5005fdc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.056740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/326] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:55.063126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 160/327] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:55.065000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.067881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.067881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.069701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:55.093393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:55.094609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:55.162924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:55.190918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/328] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:55.212527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f9c94-d5a2-40a7-a519-799d88509b06 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:55.228655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:55.266353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:55.267373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:55.268292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.268574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.268915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] PATH_INFO: `/v3/users/7a6c172094a945e1bd6d6eb519bfb6a5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.271425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] RBAC: Authorizing `identity:get_user(user_id=7a6c172094a945e1bd6d6eb519bfb6a5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:55.272940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:55.274919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 161/329] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:55 2022] GET /identity/v3/users/7a6c172094a945e1bd6d6eb519bfb6a5 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:55.282443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:55.356717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3177677-2ad4-41c0-a50a-d17f7bcbfce0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:55.416242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:55.417068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.417596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.417877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.422480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea414fb-16f6-4cb7-b3d8-76f7d1b53d80 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:55.432285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:55.443487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:55.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:55.445199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/330] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:55.450496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:55.465922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:55.467044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.467356 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.467638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] PATH_INFO: `/v3/system/users/7a6c172094a945e1bd6d6eb519bfb6a5/roles/f74f6ab86814400fb36f9d8a394c9a90` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.469966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7a6c172094a945e1bd6d6eb519bfb6a5, role_id=f74f6ab86814400fb36f9d8a394c9a90)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:55.471646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:55.475650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:55.486966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 162/331] 10.222.0.13 () {58 vars in 1334 bytes} [Tue Jul 19 12:17:55 2022] PUT /identity/v3/system/users/7a6c172094a945e1bd6d6eb519bfb6a5/roles/f74f6ab86814400fb36f9d8a394c9a90 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:17:55.600453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:55.616280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621a26d-88c5-4091-bf19-7fde8a8a6cb2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:55.658616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:55.659307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.659603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.659874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.660662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:55.660970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:55.662362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:55.671843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/332] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:55.677428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:55.807958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ac492-be22-4fce-a354-ee96b2ac0cf9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:55.844553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf2205-2082-46c6-bfce-d283475732ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:55.845899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2205-2082-46c6-bfce-d283475732ff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:55.846391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2205-2082-46c6-bfce-d283475732ff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:55.846869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2205-2082-46c6-bfce-d283475732ff None admin] PATH_INFO: `/v3/users/e34199722ba648358b9bc2fe2ba7e0cb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:55.848337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2205-2082-46c6-bfce-d283475732ff None admin] RBAC: Authorizing `identity:get_user(user_id=e34199722ba648358b9bc2fe2ba7e0cb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:55.849994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2205-2082-46c6-bfce-d283475732ff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:55.851827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 163/333] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:17:55 2022] GET /identity/v3/users/e34199722ba648358b9bc2fe2ba7e0cb => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:55.858520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:56.004401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2981f43-153f-4134-bb4f-205f4e70c991 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:56.047635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:56.048326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:56.048614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:56.048896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:56.049742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] RBAC: Adding query filter params (role.id=13dc310ce9ce466088231180b4040e69, user.id=e34199722ba648358b9bc2fe2ba7e0cb, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:56.050055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:56.051535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e78a6-7043-4c78-82bd-60eb6da60971 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:56.068337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/334] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jul 19 12:17:55 2022] GET /identity/v3/role_assignments?user.id=e34199722ba648358b9bc2fe2ba7e0cb&scope.system=all&role.id=13dc310ce9ce466088231180b4040e69 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:57.335065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfbe6d-a35a-448b-89be-c61baf3071e8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.335568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfbe6d-a35a-448b-89be-c61baf3071e8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.335665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfbe6d-a35a-448b-89be-c61baf3071e8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.337454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 164/335] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:57.353776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.354272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.354347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.374263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:57.375461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:57.379299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941114e6-d93d-4805-813d-588e5e93ccca None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.379593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941114e6-d93d-4805-813d-588e5e93ccca None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.379879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941114e6-d93d-4805-813d-588e5e93ccca None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.381551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 165/336] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:57.386424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:57.386768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.387060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.387349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.413667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:57.414882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:57.489282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/337] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:57.517562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:57.523875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 166/338] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:17:57.527609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:57.549890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:57.658574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0476df26-9c36-46d4-aefe-e1482d9479a3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:57.694750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:57.695569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.695856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.696135 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.707817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:57.717189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:57.718152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:17:57.718919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 167/339] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:17:57.724625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:57.729593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab9d94-b91e-432a-b00c-bdd311889783 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:57.788274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:57.793479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:57.794315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.794606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.794889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.817262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:57.825704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:57.828324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:57.830244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:57.836200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/340] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:57.836398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4950c1-3ccb-4386-8b36-9f8ae2d93403 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:57.837979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:57.843129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:57.876156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76370e0d-0036-4025-906c-587943c06446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:57.876792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76370e0d-0036-4025-906c-587943c06446 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:57.877083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76370e0d-0036-4025-906c-587943c06446 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:57.877368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76370e0d-0036-4025-906c-587943c06446 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:57.878216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76370e0d-0036-4025-906c-587943c06446 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:57.878599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76370e0d-0036-4025-906c-587943c06446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:57.880059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76370e0d-0036-4025-906c-587943c06446 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:57.889866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 168/341] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:57.895473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76370e0d-0036-4025-906c-587943c06446 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:58.007700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614be75-ae28-46bd-adbb-de092e5c4154 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:58.008086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76370e0d-0036-4025-906c-587943c06446 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:58.052204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:58.052204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.052204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.052204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.068666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:58.088894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:58.089662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.090030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.090394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.091220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:58.091596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:58.093126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:58.114147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/342] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:17:58.121171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:58.132391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:58.134706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:58.228740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:58.279546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42858de-bb8e-4a36-8546-ae36064a5042 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:58.319968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:58.320650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.320932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.321205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.402055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:58.403830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:58.453731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 169/343] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:58.458102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:58.504538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:58.529389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:58.633025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/344] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:17:58.641445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:58.688918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51310a36-1259-46da-8d21-3dd7e4c58877 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:58.763484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecc1fc-5c9a-4ef1-a4a6-8b34abe22ea7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:58.767238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ffe5f-194b-4862-b2cf-aede658b1a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:58.768406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ffe5f-194b-4862-b2cf-aede658b1a67 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.768903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ffe5f-194b-4862-b2cf-aede658b1a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.769367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ffe5f-194b-4862-b2cf-aede658b1a67 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.770868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ffe5f-194b-4862-b2cf-aede658b1a67 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:58.773345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ffe5f-194b-4862-b2cf-aede658b1a67 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:58.779774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 170/345] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:58 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:58.784581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97a89-f47a-46ce-8287-0988a1114a51 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.785426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97a89-f47a-46ce-8287-0988a1114a51 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.785900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c97a89-f47a-46ce-8287-0988a1114a51 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.787408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 171/346] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:58.788799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d343bd6-3077-4662-9013-0fed3e096671 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.789675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d343bd6-3077-4662-9013-0fed3e096671 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.790209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d343bd6-3077-4662-9013-0fed3e096671 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.791678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 172/347] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:58.792703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d343bd6-3077-4662-9013-0fed3e096671 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:58.829242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06728a-704d-42a4-8baa-3e1b2528cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:58.830117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06728a-704d-42a4-8baa-3e1b2528cf80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.830402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06728a-704d-42a4-8baa-3e1b2528cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.830692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06728a-704d-42a4-8baa-3e1b2528cf80 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.831896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06728a-704d-42a4-8baa-3e1b2528cf80 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:58.833393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06728a-704d-42a4-8baa-3e1b2528cf80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:58.835422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/348] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:17:58 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:58.836697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537ba73-62db-41b0-83a9-95c82c17568d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.836987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537ba73-62db-41b0-83a9-95c82c17568d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.837269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537ba73-62db-41b0-83a9-95c82c17568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.857145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e537ba73-62db-41b0-83a9-95c82c17568d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:58.858427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e537ba73-62db-41b0-83a9-95c82c17568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:58.898896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:58.910340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:58.926561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d343bd6-3077-4662-9013-0fed3e096671 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:58.946922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/349] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:58.948327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.948660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.949079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.972224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:58.972421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:58.972500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:58.972500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:58.972649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:58.972874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:58.979037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:58.980561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:58.982648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 173/350] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:17:58 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:58.983472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.071991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/351] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:59.072965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.126422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603843d-31e9-4d69-8ac6-6ba0846deebd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.164731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.165590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.165872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.166149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.167313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:59.168818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:59.171648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 174/352] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:17:58 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:59.176523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.193248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:59.204067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:59.208598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a191237-2647-4465-bb52-c66c97920bcc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.264744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.265931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.266420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.266897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.281821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:59.292414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:59.293358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:17:59.294144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:59.297426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/353] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:59.301541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.305950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca40b8-933c-4f48-832d-cb6812f55acd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.349400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.351196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.351196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.351384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.353977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:59.353977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:59.353977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:59.361157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 175/354] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=7805367da7144187ae1dd2e09705a569 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:59.361790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.438531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0274c-883a-4dfb-96b5-ec884cc255df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.476248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:59.485322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193f6e8-6718-4093-8328-bb055c41b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.486420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193f6e8-6718-4093-8328-bb055c41b26e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.486713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193f6e8-6718-4093-8328-bb055c41b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.486995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193f6e8-6718-4093-8328-bb055c41b26e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.499629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4ae9b-da77-4759-8ae8-79e188ca586c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.502357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193f6e8-6718-4093-8328-bb055c41b26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6193f6e8-6718-4093-8328-bb055c41b26e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:17:59.512283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:17:59.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:17:59.514262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:17:59.517077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/355] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:17:59.518612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f3a3f-1d9a-4e50-8d03-3dade331c3b8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.519162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f3a3f-1d9a-4e50-8d03-3dade331c3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.519659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f3a3f-1d9a-4e50-8d03-3dade331c3b8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.521209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/356] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:59.522498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f3a3f-1d9a-4e50-8d03-3dade331c3b8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.544620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:59.545050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.545855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.546142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.546421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.547342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:59.547705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:59.549255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:59.555640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 176/357] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=13dc310ce9ce466088231180b4040e69 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:59.557453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.648099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f3a3f-1d9a-4e50-8d03-3dade331c3b8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.692410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.692764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.693058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.693444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.694261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:59.694574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:59.696062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4316b3c-214c-4de6-b814-031e240a52b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:59.712132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/358] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:59.714782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c99b-f1b3-4deb-88c1-cac7cd41ef99 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.715234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c99b-f1b3-4deb-88c1-cac7cd41ef99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.715505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47c99b-f1b3-4deb-88c1-cac7cd41ef99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.740245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbecca89-afac-46ed-a171-af5a13c87cf4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:17:59.742399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bd47c99b-f1b3-4deb-88c1-cac7cd41ef99 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:59.744548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd47c99b-f1b3-4deb-88c1-cac7cd41ef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:59.787841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:17:59.788156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.788410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.788681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.789439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:17:59.790453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:17:59.791901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be009d1c-1cd8-4cb7-9ad1-64eed2e0db38 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:17:59.804997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 177/359] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:17:59.806207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f837e7-1242-4f77-add5-4f3dd4df830e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.806552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f837e7-1242-4f77-add5-4f3dd4df830e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.806785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f837e7-1242-4f77-add5-4f3dd4df830e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.808158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 178/360] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:17:59.813054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f837e7-1242-4f77-add5-4f3dd4df830e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.831434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/361] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:59.833002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:17:59.833300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:17:59.833649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:17:59.833940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:59.840289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:17:59.852789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:17:59.854066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:17:59.927308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/362] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:17:59.928286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:17:59.976861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f837e7-1242-4f77-add5-4f3dd4df830e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.013839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.014943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.015227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.015499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.037709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a6ec0-801d-4ff9-8df9-ec3f840bb7b8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.076115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.076878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.077198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.077504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.080621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.094397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.095904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.113610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.162287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.163778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.210612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 179/363] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:00.211504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.278971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/364] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:17:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:00.280424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.320452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.321406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2edf7d-21cf-49c0-ac38-62b254594d7c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.336213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.371626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.372460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.372743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.373020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.386521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:00.397053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:00.398070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 19 12:18:00.399061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:00.400216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 180/365] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:00.404201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.407669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e21873-6dde-4e7d-ba11-4fa52256af04 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.452141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.453008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.453298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.453579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.465254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:00.474267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:00.475139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:00.475911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/366] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:00.477059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.544481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021db16-1023-42e0-90ff-348bcc6ddb76 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.587537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.588517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.588807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.589134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.590409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.591954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.594542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 181/367] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:00.599617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.622290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98935ba-d123-408b-a6de-689569e12492 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.628513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.656231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.671337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.672224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.672517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.672791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.674676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.676531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.683953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/368] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:00.685343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.741039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c6971-d8ed-4b04-8f08-1c4b3eb1ff95 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.786350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.787032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.787325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.787610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.788393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:00.788713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.790492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.799129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 182/369] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:00.800716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.826070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa99a78-adf3-4472-91af-3acbbde1ab83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.871058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.872399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.872897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.873729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.875238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:00.875834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.878027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.888788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/370] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:00.891364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:00.922804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.925844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:00.932450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7538-8ca9-4357-b7fc-ce088e70ae21 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:00.976054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:00.977118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:00.977484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:00.977843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:00.979128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:00.980911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:00.982443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 183/371] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:00.983479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.023194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ea16-9899-4fd6-a617-b3a61a6f4576 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.067335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.068551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.069471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.069930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.071750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.074001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.076003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/372] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.077482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.107015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3419eaf-7aad-415f-8e10-4defde92746b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.146097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.147138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.147510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.147861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] PATH_INFO: `/v3/users/7a6c172094a945e1bd6d6eb519bfb6a5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.149181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] RBAC: Authorizing `identity:get_user(user_id=7a6c172094a945e1bd6d6eb519bfb6a5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.150740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.155179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 184/373] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:18:00 2022] GET /identity/v3/users/7a6c172094a945e1bd6d6eb519bfb6a5 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.156124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.188104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:01.195208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:01.200060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c693e-fede-4cd8-8d9a-15b5668b4be0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.235406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.236374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.236834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.237301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.274372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302d817-1a0d-47ce-956b-7b5054f86f61 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.311609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.312024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.312814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.313175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.313533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.314272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.314434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=3fc57ef585554a139d14be6c444c334f) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:01.314872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.316463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.323357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 185/374] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=13dc310ce9ce466088231180b4040e69 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.324295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.415530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:01.427401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/375] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:01.428745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.445258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:01.449829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd50ab-95cf-4f31-8ed4-a8c7911b9697 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.492946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.494276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.494769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.495241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.496301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=3fc57ef585554a139d14be6c444c334f) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:01.496888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.498784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.505978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 186/376] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=7805367da7144187ae1dd2e09705a569 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.508015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.539937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27638dd0-231f-43dd-93d8-5606c4485510 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.579827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.580613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.581006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.581366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.582291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] RBAC: Adding query filter params (role.id=f74f6ab86814400fb36f9d8a394c9a90, user.id=7a6c172094a945e1bd6d6eb519bfb6a5, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:01.582716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.584484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.604223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/377] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/role_assignments?user.id=7a6c172094a945e1bd6d6eb519bfb6a5&scope.system=all&role.id=f74f6ab86814400fb36f9d8a394c9a90 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.645719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b3639-36a1-4803-9468-cd3355941403 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.687641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9482355-6310-4d42-94d3-072d434bf236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.688927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9482355-6310-4d42-94d3-072d434bf236 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.689429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9482355-6310-4d42-94d3-072d434bf236 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.689910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9482355-6310-4d42-94d3-072d434bf236 None admin] PATH_INFO: `/v3/projects/45290f5162bc4ec4b133291838122f88` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.691485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9482355-6310-4d42-94d3-072d434bf236 None admin] RBAC: Authorizing `identity:get_project(project_id=45290f5162bc4ec4b133291838122f88)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.693233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9482355-6310-4d42-94d3-072d434bf236 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.695254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 187/378] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/projects/45290f5162bc4ec4b133291838122f88 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.701303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.773858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:01.835179 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde2e48-304e-40c6-9d26-35093ba05d83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:01.856881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fbf6-95de-48b7-b16a-0353a93211d6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.857240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fbf6-95de-48b7-b16a-0353a93211d6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.857445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fbf6-95de-48b7-b16a-0353a93211d6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.859132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 188/379] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:01.867982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.868204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.868478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.880288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:01.881165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:01.881452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:01.881727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] PATH_INFO: `/v3/groups/363a92567d4a4bf8ab111063a471fa08` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:01.897748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:01.898968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:01.901760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] RBAC: Authorizing `identity:get_group(group_id=363a92567d4a4bf8ab111063a471fa08)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:01.903231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:01.905785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/380] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/groups/363a92567d4a4bf8ab111063a471fa08 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:01.911238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:01.958692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:02.042562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 189/381] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:02.067543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.081952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ae6ba-3ed5-4754-9cd5-55083e508729 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.099248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:02.133714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.134388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.134678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.134963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.135811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] RBAC: Adding query filter params (group.id=363a92567d4a4bf8ab111063a471fa08, role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=45290f5162bc4ec4b133291838122f88) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:02.136131 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:02.138898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81280343-94fc-45a7-87cb-be202e341b86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:02.145444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/382] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:01 2022] GET /identity/v3/role_assignments?group.id=363a92567d4a4bf8ab111063a471fa08&scope.project.id=45290f5162bc4ec4b133291838122f88&role.id=ac904f211a8f450fbf556c879296af4b => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:02.247432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f14b7-b2a4-464e-af2e-b5c69da1fa67 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.247832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f14b7-b2a4-464e-af2e-b5c69da1fa67 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.248012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f14b7-b2a4-464e-af2e-b5c69da1fa67 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.249790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/383] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:02.255301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.255589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.255866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.256098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a206a45-b30c-4050-bd37-791f4d9d3b19 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.275558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:02.276817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:02.304211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.305147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.305537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.305819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.322009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:02.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:02.335513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:02.336868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 190/384] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:02.342080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.355671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/385] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:02.375221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:02.378681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.488255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3dbdc3-6c82-4cb1-ba79-6588ed9986fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.500710 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b196aba-ff7d-4a91-9fb7-c3d3333305d2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.503518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:02.540901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d07074-e513-4892-a033-91f966c48214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.541694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d07074-e513-4892-a033-91f966c48214 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.541975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d07074-e513-4892-a033-91f966c48214 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.542246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d07074-e513-4892-a033-91f966c48214 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.565995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.566671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.566954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.567229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.567983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:02.568315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:02.568955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d07074-e513-4892-a033-91f966c48214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:02.576229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-92d07074-e513-4892-a033-91f966c48214 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:02.579837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:02.580741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:02.582177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/386] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:02.588976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 191/387] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:02.589786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d07074-e513-4892-a033-91f966c48214 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.613880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.737367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d07074-e513-4892-a033-91f966c48214 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.755201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:02.780909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989e1fe-8465-48d9-8c02-824d679695f6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.784794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.785462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.785746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.786023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.786790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:02.787101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:02.793102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:02.807171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/388] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:02.814398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.853543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:02.873876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.874745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.874965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.875238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.876910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:02.878364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:02.879989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 192/389] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:02.889720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:02.939144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ee7e9-e494-4080-80ce-4791690e2423 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:02.983592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:02.984871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:02.985377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:02.985863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:02.987926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:02.989911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:02.996960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/390] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:02.999877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:03.028645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13728a80-e720-4a35-abda-01628b19c4cb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:03.072747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:03.073625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:03.073914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:03.074285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:03.076321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:03.078340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:03.080266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 193/391] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:03.085524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:03.128660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:03.128835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:03.138378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97306d99-1c2b-4d8a-9b82-316f915dc9c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:03.187811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:03.189114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:03.189114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:03.189292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:03.190786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:03.192457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:03.195356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/392] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:02 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:03.203895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:03.235113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23e551-a5df-4ad0-8fad-dd940a938cd9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:03.289069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796a6d9-676f-40e3-a0d8-150ef84458a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:03.289837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a6d9-676f-40e3-a0d8-150ef84458a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:03.290184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a6d9-676f-40e3-a0d8-150ef84458a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:03.290398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796a6d9-676f-40e3-a0d8-150ef84458a0 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/groups/aa9c4967f398491794849777eb138243/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:03.292922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796a6d9-676f-40e3-a0d8-150ef84458a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2df5ade771f42f78fe0ab403129aa5d, group_id=aa9c4967f398491794849777eb138243, role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:03.294385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796a6d9-676f-40e3-a0d8-150ef84458a0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:03.323539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 194/393] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jul 19 12:18:03 2022] PUT /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/groups/aa9c4967f398491794849777eb138243/roles/7805367da7144187ae1dd2e09705a569 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:03.351613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c0de-9cd9-426b-82e6-00a6e2ee9249 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:03.399846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c71a9-c425-478e-b311-80e1b8a930cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:03.401004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c71a9-c425-478e-b311-80e1b8a930cf None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:03.401431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c71a9-c425-478e-b311-80e1b8a930cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:03.401874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c71a9-c425-478e-b311-80e1b8a930cf None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/groups/aa9c4967f398491794849777eb138243/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:03.403981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c71a9-c425-478e-b311-80e1b8a930cf None admin] RBAC: Authorizing `identity:create_grant(project_id=d2df5ade771f42f78fe0ab403129aa5d, group_id=aa9c4967f398491794849777eb138243, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:03.406116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c71a9-c425-478e-b311-80e1b8a930cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:03.428319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/394] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jul 19 12:18:03 2022] PUT /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d/groups/aa9c4967f398491794849777eb138243/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:04.171914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd006-0529-4efc-9064-7c0a79ba4860 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.172478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd006-0529-4efc-9064-7c0a79ba4860 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.172554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afd006-0529-4efc-9064-7c0a79ba4860 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.174250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 195/395] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:04.181175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfa4dc-6e17-45c6-a449-5cfbd6e9dec0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.181668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfa4dc-6e17-45c6-a449-5cfbd6e9dec0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.181757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfa4dc-6e17-45c6-a449-5cfbd6e9dec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.212067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-93bfa4dc-6e17-45c6-a449-5cfbd6e9dec0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:04.213323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93bfa4dc-6e17-45c6-a449-5cfbd6e9dec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:04.241156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:04.306456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/396] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:04.352053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd006-0529-4efc-9064-7c0a79ba4860 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:04.453881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:04.464509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d133e-f1a2-43f9-a9bf-b11e749a1a53 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.464799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d133e-f1a2-43f9-a9bf-b11e749a1a53 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.465079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d133e-f1a2-43f9-a9bf-b11e749a1a53 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.466581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/397] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:04.473504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.473792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.474075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.500335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:04.501528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:04.506211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afd006-0529-4efc-9064-7c0a79ba4860 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:04.564713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:04.565895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.566325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.566760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.579637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:04.588711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:04.589607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:04.590475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:04.593190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 196/398] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:04.599538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:04.654542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/399] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:04.686614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:04.731900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b832786-9291-41e7-b9d6-ef4da7e1d872 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:04.773342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:04.774288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.774715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.775134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.776531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] RBAC: Adding query filter params (name=member) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:04.776986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:04.779181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:04.796621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 197/400] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:18:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:04.798246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:04.801819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:04.837392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:04.868252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e57dc-2b81-4852-b33f-1afa7d44904b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:04.942611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:04.943428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:04.943709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:04.943984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:04.958276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:04.971491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:04.972624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:04.973493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/401] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:04.981575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.004864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fc31c-c4a0-43d7-8f76-be595d30b85b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.042578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dcc11-d137-4cc0-b179-155c291ac3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.043441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcc11-d137-4cc0-b179-155c291ac3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.043723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcc11-d137-4cc0-b179-155c291ac3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.044002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcc11-d137-4cc0-b179-155c291ac3f6 None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.045440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dcc11-d137-4cc0-b179-155c291ac3f6 None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.046886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591dcc11-d137-4cc0-b179-155c291ac3f6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.048989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 198/402] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:04 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:05.050172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162c4ee-a6e5-460a-be1b-de341ea914e5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.050457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162c4ee-a6e5-460a-be1b-de341ea914e5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.050733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162c4ee-a6e5-460a-be1b-de341ea914e5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.052166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 199/403] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:05.056868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162c4ee-a6e5-460a-be1b-de341ea914e5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.097526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d33fb-5b74-409c-8345-4f58c07d0a09 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.136835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:05.148088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.148892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.149180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.149458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.150323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:05.150635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.152034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bec63c-9ee6-4a66-93e1-06d6ed659f49 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.167516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/404] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:05.169063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.169355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.169637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.181962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162c4ee-a6e5-460a-be1b-de341ea914e5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.194969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:05.196491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:05.198701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:05.221643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.223060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.223484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.223774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.225157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.227251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.230059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 200/405] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:05 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:05.230914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.273609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/406] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:05.280443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.355647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e69839-73fe-4384-9ae2-f0017d01978a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.392511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.393462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.393748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.394113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.395279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.396851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.403045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 201/407] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:05 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:05.403979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.410255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75612d-9819-4cbc-a50b-60d19500f4bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.413545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:05.452662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.453864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.454435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.454776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f/groups/aa9c4967f398491794849777eb138243/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.457367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] RBAC: Authorizing `identity:create_grant(project_id=3fc57ef585554a139d14be6c444c334f, group_id=aa9c4967f398491794849777eb138243, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.459089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.469633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:05.479160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/408] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jul 19 12:18:05 2022] PUT /identity/v3/projects/3fc57ef585554a139d14be6c444c334f/groups/aa9c4967f398491794849777eb138243/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:05.480605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.532807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ba211-3b2a-4278-b894-3d01382d14df None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.587564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.588455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.588750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.590757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.618360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:05.631163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:05.633355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:05.635270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 202/409] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:05.641046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0005f70-4c39-4c2d-b97c-b321e82bedbd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.643621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.718444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.720953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.721462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.722009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.723540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.725332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.727591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/410] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:05 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:05.732990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.735641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:05.779103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278946c-d10a-4b04-896b-20385877dc01 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.816728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:05.823561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.824267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.824554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.824830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.825595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:05.825900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.827472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.844676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 203/411] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:05.849100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:05.871310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d87b0f-39da-4d33-ba36-00a72da45658 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:05.918540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:05.920971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:05.921630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:05.922313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f/groups/aa9c4967f398491794849777eb138243/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:05.925999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fc57ef585554a139d14be6c444c334f, group_id=aa9c4967f398491794849777eb138243, role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:05.928132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:05.945922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/412] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jul 19 12:18:05 2022] PUT /identity/v3/projects/3fc57ef585554a139d14be6c444c334f/groups/aa9c4967f398491794849777eb138243/roles/7805367da7144187ae1dd2e09705a569 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:06.015966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38a015-9099-48d8-b31b-f4f5fc6c0f31 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.051328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:06.066896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e920d5f-94bd-42c5-b995-933fd4e74ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.067707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e920d5f-94bd-42c5-b995-933fd4e74ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.067993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e920d5f-94bd-42c5-b995-933fd4e74ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.068319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e920d5f-94bd-42c5-b995-933fd4e74ed0 None admin] PATH_INFO: `/v3/groups/363a92567d4a4bf8ab111063a471fa08` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.069546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e920d5f-94bd-42c5-b995-933fd4e74ed0 None admin] RBAC: Authorizing `identity:get_group(group_id=363a92567d4a4bf8ab111063a471fa08)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:06.071067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e920d5f-94bd-42c5-b995-933fd4e74ed0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:06.073761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 204/413] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:05 2022] GET /identity/v3/groups/363a92567d4a4bf8ab111063a471fa08 => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:06.078754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.122412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9049-08ff-4de2-88b6-182a4ee069dc None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.122735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9049-08ff-4de2-88b6-182a4ee069dc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.122994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9049-08ff-4de2-88b6-182a4ee069dc None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.124776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 205/414] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:06.130020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.130315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.130601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.155979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:06.157333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:06.209832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dfa4b-a567-4969-b286-56e166d0a1b9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.248649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 206/415] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:06.262309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:06.274540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.280111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025d457-f0cd-4597-a804-596964282087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.281169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025d457-f0cd-4597-a804-596964282087 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.281463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025d457-f0cd-4597-a804-596964282087 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.281745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025d457-f0cd-4597-a804-596964282087 None admin] PATH_INFO: `/v3/projects/45290f5162bc4ec4b133291838122f88` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.282975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025d457-f0cd-4597-a804-596964282087 None admin] RBAC: Authorizing `identity:get_project(project_id=45290f5162bc4ec4b133291838122f88)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:06.285029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025d457-f0cd-4597-a804-596964282087 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:06.287431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/416] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:06 2022] GET /identity/v3/projects/45290f5162bc4ec4b133291838122f88 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:06.289050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4743-ec9c-483f-9816-935dec42d51a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.289348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4743-ec9c-483f-9816-935dec42d51a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.289636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4743-ec9c-483f-9816-935dec42d51a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.290868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/417] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:06.293706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4743-ec9c-483f-9816-935dec42d51a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.390643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b386b3-ddce-409f-8f62-867966bf6198 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.417717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:06.418703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4743-ec9c-483f-9816-935dec42d51a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.428085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42198c4c-0dbe-4ef8-840e-d25ba7d908b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.428953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42198c4c-0dbe-4ef8-840e-d25ba7d908b5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.429304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42198c4c-0dbe-4ef8-840e-d25ba7d908b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.429579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42198c4c-0dbe-4ef8-840e-d25ba7d908b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.441591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42198c4c-0dbe-4ef8-840e-d25ba7d908b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-42198c4c-0dbe-4ef8-840e-d25ba7d908b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:06.450147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:06.451005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:06.451874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:06.452488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 207/418] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:06.455365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.455365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.455365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.464367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.465253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.465531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.465803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] PATH_INFO: `/v3/projects/45290f5162bc4ec4b133291838122f88/groups/363a92567d4a4bf8ab111063a471fa08/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.469219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] RBAC: Authorizing `identity:create_grant(project_id=45290f5162bc4ec4b133291838122f88, group_id=363a92567d4a4bf8ab111063a471fa08, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:06.470861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:06.472486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:06.473927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:06.488495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/419] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Jul 19 12:18:06 2022] PUT /identity/v3/projects/45290f5162bc4ec4b133291838122f88/groups/363a92567d4a4bf8ab111063a471fa08/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:06.489444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.581243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:06.581534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 208/420] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:06.627309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.634382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81828af-886c-458e-8298-2a7d2972e466 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.675643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.676266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.676558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.676840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.677896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:06.678214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:06.679600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:06.689319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/421] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:18:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:06.695462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.817804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf9af2-1892-49b5-beab-ddf2eaf4ea39 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.823018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8070b0d-6bed-410d-847c-c27e23263a4b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:06.848753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:06.853586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:06.869115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.870089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.870574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.871072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.890420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:06.891337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:06.891625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:06.891899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:06.915154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:06.924744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:06.925628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:06.926446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 209/422] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:06.933223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:06.946723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:06.948453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.076507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cfdc3-c866-4e64-a438-9d6b96489eee None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:07.089433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/423] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:07.095419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:07.114369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:07.115054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:07.115342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:07.115624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:07.116421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:07.116732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:07.118177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.128050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 210/424] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:07.130055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:07.133530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:07.183185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:07.252527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ea567-69ea-4a51-b308-0ece1f9070ac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:07.294585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:07.296332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:07.296620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:07.296901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:07.358099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53939-acb9-4ea4-9e8c-5573456dc476 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:07.379250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:07.381234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.424683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:07.447843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:07.448751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:07.449037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:07.449313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:07.457076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:07.458712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.460850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/425] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:07.466188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:07.494255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 211/426] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:07.499567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:07.631378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a3884-24b6-44c6-a0b8-5551d5ab23b5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:07.668211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:07.670029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:07.670322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:07.670607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] PATH_INFO: `/v3/projects/d2df5ade771f42f78fe0ab403129aa5d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:07.671866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] RBAC: Authorizing `identity:get_project(project_id=d2df5ade771f42f78fe0ab403129aa5d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:07.673352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.675386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 212/427] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/projects/d2df5ade771f42f78fe0ab403129aa5d => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:07.683328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:07.699392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:07.699882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:07.740521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ca268-9e1e-40d7-9c3b-e7e4dbc650a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:07.806768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:07.807638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:07.807921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:07.812290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:07.813228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:07.816134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.822093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/428] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:07.828291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:07.851266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6cc65-01e9-496c-8ab4-32ff7f53bc5c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:07.923331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:07.924217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:07.924540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:07.924818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:07.926650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:07.928105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:07.930187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 213/429] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:07.935733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:08.022464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:08.052709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482103b5-6876-4e79-acef-77481dc37b07 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.086088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3e639-230b-4ec2-a6a8-f8ef77e5bd60 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.091870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.092836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.093339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.093813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.094848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:08.095784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:08.097488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651537fd-a920-4a7b-a220-288e34743709 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:08.104982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 214/430] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=7805367da7144187ae1dd2e09705a569 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:08.117831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8555c-b4f0-4aea-a8e9-09225dcaa95f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.118162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8555c-b4f0-4aea-a8e9-09225dcaa95f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.119076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8555c-b4f0-4aea-a8e9-09225dcaa95f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.120518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 215/431] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:08.126638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5139b-861e-4c90-8971-b5f50ee09e24 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.126956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5139b-861e-4c90-8971-b5f50ee09e24 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.127202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5139b-861e-4c90-8971-b5f50ee09e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.145684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2be5139b-861e-4c90-8971-b5f50ee09e24 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:08.146893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2be5139b-861e-4c90-8971-b5f50ee09e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:08.166900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.167572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.167858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.168133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.169014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=d2df5ade771f42f78fe0ab403129aa5d) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:08.169326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:08.171404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:08.190886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/432] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:07 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=d2df5ade771f42f78fe0ab403129aa5d&role.id=13dc310ce9ce466088231180b4040e69 => generated 611 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:08.226660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 216/433] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:08.251567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:08.295420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:08.366234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4886c3e-80fb-4f21-b48a-e637af01a5aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.406869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.407695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.407987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.408295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.414015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5d57e-1347-47e5-8887-9d14c23c898e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.414513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5d57e-1347-47e5-8887-9d14c23c898e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.414599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5d57e-1347-47e5-8887-9d14c23c898e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.416213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 217/434] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:08.422933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.423492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.423976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.425109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:08.437049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:08.443545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/435] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:08.444379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:08.449404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:08.451152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:08.524886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:08.533108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 218/436] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:08.538189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:08.557867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:08.565477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cf99b-0114-40a3-9fb0-3bd243b6e54b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.602266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.602968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.603259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.603543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.604700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:08.604700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:08.606121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:08.615516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/437] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:18:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:08.621341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:08.701706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a37be-932e-47ca-aa5a-0cb7561d03ff None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.745832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.746992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.747468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.748545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.757021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be089cc-54f8-439a-beed-bb9e54a51361 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.759535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:08.769102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:08.769995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:08.770758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 219/438] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:08.777193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:08.809009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0849e-b749-47b9-b99b-f86298933798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.809889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0849e-b749-47b9-b99b-f86298933798 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.810323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0849e-b749-47b9-b99b-f86298933798 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.811273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0849e-b749-47b9-b99b-f86298933798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.822686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:08.851188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:08.892506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0849e-b749-47b9-b99b-f86298933798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:08.894218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0849e-b749-47b9-b99b-f86298933798 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:08.911883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67004e-aeba-40fb-ac1f-3b0b48e123f8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:08.949579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:08.950254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:08.950540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:08.950822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:08.952101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:08.953927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:08.954922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:08.964570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 220/439] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:08.970068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.010588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/440] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:18:09.011764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4433a-fafc-49d7-8658-8d1a21a7d70a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.012062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4433a-fafc-49d7-8658-8d1a21a7d70a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.012377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4433a-fafc-49d7-8658-8d1a21a7d70a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.016228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/441] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:09.016228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4433a-fafc-49d7-8658-8d1a21a7d70a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.055905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.073366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd324bb5-0846-4fde-9196-65eb9e5fc42d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.093991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.108535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.109173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.109459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.109731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.119632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4433a-fafc-49d7-8658-8d1a21a7d70a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.153414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6391933-a642-40cd-ade0-a3e38ffb31b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.154334 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391933-a642-40cd-ade0-a3e38ffb31b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.154621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391933-a642-40cd-ade0-a3e38ffb31b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.154900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6391933-a642-40cd-ade0-a3e38ffb31b8 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.156069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6391933-a642-40cd-ade0-a3e38ffb31b8 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.157525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6391933-a642-40cd-ade0-a3e38ffb31b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.159200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/442] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:09.163669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.163669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.163669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.176803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:09.177986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:09.180079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.181583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.244758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/443] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:09.245413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.281829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.282588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 221/444] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:18:09.283565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.307291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.348523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421c49d-9e11-404e-b62f-2e0044885dd8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.382131 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.382978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.383257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.383529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.384143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ca58-2afd-44b0-bb9d-2bf322219d58 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.384721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.386147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.387747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/445] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:09.388568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.419875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.420693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.420980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.421264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.431167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:09.439286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:09.440143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:09.440964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 222/446] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:09.441589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.489367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02dfa3-2af0-4401-9d4d-9bf457600a76 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.491710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.519836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.523966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.524850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.525137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.525411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] PATH_INFO: `/v3/projects/3fc57ef585554a139d14be6c444c334f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.526597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] RBAC: Authorizing `identity:get_project(project_id=3fc57ef585554a139d14be6c444c334f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.528023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.530266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/447] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/projects/3fc57ef585554a139d14be6c444c334f => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:09.531025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.543670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0c64a-2cb1-4fd2-abc5-675aaf7f7d44 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.580130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.581149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.581590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.582022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.582931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=13dc310ce9ce466088231180b4040e69, scope.project.id=3fc57ef585554a139d14be6c444c334f) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:09.583254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.584752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.590737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 223/448] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=13dc310ce9ce466088231180b4040e69 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:09.591509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.638236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00dd1f-e143-4741-9d18-b6126ce44472 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.678438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.679145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.679394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.679672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.680573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:09.680887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.682313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.696056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/449] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:09.697431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.716371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df488b4-4bca-48b0-aa5f-cf1d7ff74900 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.736103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.756155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.756155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.756464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.756743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] PATH_INFO: `/v3/groups/aa9c4967f398491794849777eb138243` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.757935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] RBAC: Authorizing `identity:get_group(group_id=aa9c4967f398491794849777eb138243)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.759410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.761053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/groups/aa9c4967f398491794849777eb138243 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:09.767880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:09.785593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:09.818580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50eafc-fa25-4391-9c01-170a70ae23c3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.859314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.860378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.860683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.860964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.891263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456991d-642b-4766-8965-dc5ef10fcacb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:09.932750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:09.933569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:09.933854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:09.934136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:09.934977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] RBAC: Adding query filter params (group.id=aa9c4967f398491794849777eb138243, role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=3fc57ef585554a139d14be6c444c334f) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:09.935291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.937359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.943472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 225/451] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/role_assignments?group.id=aa9c4967f398491794849777eb138243&scope.project.id=3fc57ef585554a139d14be6c444c334f&role.id=7805367da7144187ae1dd2e09705a569 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:09.946950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:09.949268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:09.986829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:10.057310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/452] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:10.063199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:10.142717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:10.171565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65735ff8-d993-49b1-ae4f-231900173e40 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:10.209566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:10.210438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:10.210727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:10.211007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] PATH_INFO: `/v3/projects/45290f5162bc4ec4b133291838122f88` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:10.212230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] RBAC: Authorizing `identity:get_project(project_id=45290f5162bc4ec4b133291838122f88)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:10.213692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:10.215402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 226/453] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:18:10 2022] GET /identity/v3/projects/45290f5162bc4ec4b133291838122f88 => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:10.221806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:10.323110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adc40c-764a-4201-99f2-990c5d5d49af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:10.356033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f2425-8b1e-4088-837a-bed7579866d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:10.356901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2425-8b1e-4088-837a-bed7579866d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:10.357192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2425-8b1e-4088-837a-bed7579866d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:10.358047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2425-8b1e-4088-837a-bed7579866d7 None admin] PATH_INFO: `/v3/groups/363a92567d4a4bf8ab111063a471fa08` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:10.359219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f2425-8b1e-4088-837a-bed7579866d7 None admin] RBAC: Authorizing `identity:get_group(group_id=363a92567d4a4bf8ab111063a471fa08)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:10.360694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f2425-8b1e-4088-837a-bed7579866d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:10.362310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/454] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Jul 19 12:18:10 2022] GET /identity/v3/groups/363a92567d4a4bf8ab111063a471fa08 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:10.368326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:10.464618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aea033-8e40-4a69-a35d-866c1fa1b9ec None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:10.496738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:10.497383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:10.497669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:10.497948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:10.498770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] RBAC: Adding query filter params (group.id=363a92567d4a4bf8ab111063a471fa08, role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=45290f5162bc4ec4b133291838122f88) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:10.499082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:10.500544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b825c-24a9-46e3-bc6b-f7d9712ec5b3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:10.506271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 227/455] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Jul 19 12:18:10 2022] GET /identity/v3/role_assignments?group.id=363a92567d4a4bf8ab111063a471fa08&scope.project.id=45290f5162bc4ec4b133291838122f88&role.id=ac904f211a8f450fbf556c879296af4b => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:13.082268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c214-273d-4c12-bb31-fd2b85ff14de None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.082812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c214-273d-4c12-bb31-fd2b85ff14de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.082812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2c214-273d-4c12-bb31-fd2b85ff14de None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.084482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/456] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:13.089643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.090040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.090251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.114697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:13.115890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:13.117990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:13.171932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0ff6-d4d5-4101-80ad-2fd7dbf0b5d2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.172612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0ff6-d4d5-4101-80ad-2fd7dbf0b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.173103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0ff6-d4d5-4101-80ad-2fd7dbf0b5d2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.174906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/457] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:13.179935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.180501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.180983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.186798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:13.202429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:13.203953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:13.206716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 228/458] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:13.233590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.308989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/459] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:13.346295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.406219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792b75d-baea-48e4-87a7-86357236d05c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:13.446148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:13.447654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.448163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.449063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.454732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f602c1-5394-4531-b04a-9fd290db3ec9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:13.459110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:13.463968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:13.466075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:13.488395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2a7ceffc-4e54-4286-8519-5773ff8ccb87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:13.492497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:13.493351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:13.494416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:13.494416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:13.494416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:13.494416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:13.497476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:13.498588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.499064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 229/460] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:13.499262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.499738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.500409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f14d8-65c3-4128-8cc5-b2e6866a073f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.500700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f14d8-65c3-4128-8cc5-b2e6866a073f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.501144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f14d8-65c3-4128-8cc5-b2e6866a073f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.502792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 230/461] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:13.507264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f14d8-65c3-4128-8cc5-b2e6866a073f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.519821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:13.522297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-32816543-8356-4b6a-96d4-4ce6504d00f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:13.547959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:13.548867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:13.549693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:13.549693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:13.549693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:13.549693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:13.552453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/462] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:13.554452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.554946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.555427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.577843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:13.579663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:13.623423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f14d8-65c3-4128-8cc5-b2e6866a073f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:13.667546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/463] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:13.668656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.673785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:13.674602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.674981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.675377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.676265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:13.677465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:13.683177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:13.702542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 231/464] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:13.703382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.758616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:13.798901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0edd8c-60f0-480a-9945-16b2ad0d7736 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:13.811328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c413b-5493-486f-93e8-5f7dd8a00f25 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:13.814562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:13.837483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:13.838235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.838428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.838706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.839423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:13.839728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:13.841141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:13.848130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:13.849175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.849635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.850086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.859175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/465] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:13.859973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.866342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:13.867999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c7713954-1b43-4dec-82dc-d1076c093317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:13.886674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:13.887555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:13.888694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:13.888694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:13.888694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:13.888694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:13.890254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 232/466] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:13.891649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56539887-7fb6-49da-8814-126c864eba57 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:13.891987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56539887-7fb6-49da-8814-126c864eba57 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:13.892305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56539887-7fb6-49da-8814-126c864eba57 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:13.893708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 233/467] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:13.897294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56539887-7fb6-49da-8814-126c864eba57 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:13.971050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c35627-61a7-493a-8449-088b0ba705e2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.006733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.007404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.007694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.007976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.008726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.010184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.015873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56539887-7fb6-49da-8814-126c864eba57 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.017513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:14.031744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/468] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:18:13 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:14.041659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.056309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030557d2-3e7e-4fcd-9039-f8cca9284518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.057011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030557d2-3e7e-4fcd-9039-f8cca9284518 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.057293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030557d2-3e7e-4fcd-9039-f8cca9284518 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.057573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030557d2-3e7e-4fcd-9039-f8cca9284518 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.058362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030557d2-3e7e-4fcd-9039-f8cca9284518 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.060056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030557d2-3e7e-4fcd-9039-f8cca9284518 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.085882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 234/469] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:18:13 2022] POST /identity/v3/users => generated 248 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:14.086534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:14.089181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.089676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.090559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.113807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:14.115380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:14.172822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d0117-4ff3-405e-963e-01c4836a5c0b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.198712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 235/470] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:14.226837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.244228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.247270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.247925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.249485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.252141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:14.252667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.256415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.279776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/471] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:14.287040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.359353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:14.373807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7200938-bd75-4da4-8c80-e3bf29bf9612 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.403950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:14.411355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6552e-422b-4229-bd50-0c06a118aab3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.417155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.417964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.418241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.418515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.431738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.433153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.449092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992713d0-a262-4ea1-b2a8-ad3ca2de9a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.449760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992713d0-a262-4ea1-b2a8-ad3ca2de9a35 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.450042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992713d0-a262-4ea1-b2a8-ad3ca2de9a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.450319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992713d0-a262-4ea1-b2a8-ad3ca2de9a35 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.451558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992713d0-a262-4ea1-b2a8-ad3ca2de9a35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.453026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992713d0-a262-4ea1-b2a8-ad3ca2de9a35 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-fa43f749-b0e5-4f5d-af60-5762d81bee2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:14.454061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:14.455045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:14.456084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:14.456084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:14.456084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:14.456084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:14.456084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 236/472] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:14.456889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e8021-f8b2-48d3-abea-fbbb04eb779e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.457179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e8021-f8b2-48d3-abea-fbbb04eb779e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.457461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e8021-f8b2-48d3-abea-fbbb04eb779e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.458993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 237/473] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:14.465367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e8021-f8b2-48d3-abea-fbbb04eb779e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.486965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/474] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:18:14 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:14.490176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.490565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.490847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.523335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:14.524610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:14.674235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e8021-f8b2-48d3-abea-fbbb04eb779e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.677290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/475] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:14.713160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.718420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.719384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.719916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.720610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.721679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:14.722190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.723789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.734978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:14.739623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 238/476] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:14.745002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.811446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:14.849170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f0fc0-cb55-47a5-9c02-26204839b4df None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.887967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91e40b-760f-441d-87dc-201b3caf7469 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:14.891637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.892538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.892984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.893398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.894288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.895855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.903846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 12:18:14.903846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 12:18:14.903846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [parameters: {'id': '2284bb4cd8e1452f9a5dedd26d2f94cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 12:18:14.903846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 19 12:18:14.926606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 12:18:14.927800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 12:18:14.928984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 12:18:14.930016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 12:18:14.931072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 12:18:14.932041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': '2284bb4cd8e1452f9a5dedd26d2f94cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:14.933043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 19 12:18:14.935116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:14.939621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/477] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jul 19 12:18:14 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:14.941770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:14.942638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:14.943080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:14.943512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:14.944474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:14.946102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:14.947196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:14.996001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 239/478] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:18:14 2022] POST /identity/v3/users => generated 249 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:18:15.074016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb86ef0-4c48-4c6f-9835-cd054d429fe9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:15.112786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3599f1-f975-4553-a2ea-7dbf798daa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:15.114405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3599f1-f975-4553-a2ea-7dbf798daa44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:15.114899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3599f1-f975-4553-a2ea-7dbf798daa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:15.115372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3599f1-f975-4553-a2ea-7dbf798daa44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:15.125270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:15.130221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3599f1-f975-4553-a2ea-7dbf798daa44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ba3599f1-f975-4553-a2ea-7dbf798daa44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:15.146183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:15.147133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 19 12:18:15.147982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:15.150600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/479] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:15.157365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:15.241534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:15.296375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17ddc7-03ae-4269-87d1-f0f26e4b71a6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:15.339475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:15.340114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:15.340423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:15.340702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:15.341978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:15.342285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:15.343854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6b0323-a37f-416a-b33f-f2255199085b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:15.352765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 240/480] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:16.631263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746ceb-0446-4705-b20b-de9b7ba73a71 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:16.631723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746ceb-0446-4705-b20b-de9b7ba73a71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:16.631923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d746ceb-0446-4705-b20b-de9b7ba73a71 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:16.633523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/481] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:16.639396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:16.639806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:16.639991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:16.663273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:16.664517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:16.694203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b5419-56a9-4b0b-9c1b-5c6d4cbc4236 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:16.694203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b5419-56a9-4b0b-9c1b-5c6d4cbc4236 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:16.694203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b5419-56a9-4b0b-9c1b-5c6d4cbc4236 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:16.694874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/482] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:16.699986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:16.700539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:16.701602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:16.733643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:16.735162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:16.778611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 241/483] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:16.821233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:16.824524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/484] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:16.865191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:16.893540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:16.945855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec92e02-ac0d-4cf6-94f4-9f20db5e53e0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:16.982930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:16.984093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:16.984695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:16.985161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:16.996317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:17.004925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:17.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:17.006607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:17.009372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 242/485] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:17.017606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.099917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7323dc3a-1f59-47ba-89f6-999ff7108ffe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.159683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.171776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec0353-ef87-4e3c-9ab0-49d1d94eace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.173442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b256f-a2b3-453f-8cc9-b5ae638a6a61 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.173562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec0353-ef87-4e3c-9ab0-49d1d94eace2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.173969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec0353-ef87-4e3c-9ab0-49d1d94eace2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.174350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec0353-ef87-4e3c-9ab0-49d1d94eace2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.184034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.187657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec0353-ef87-4e3c-9ab0-49d1d94eace2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b1ec0353-ef87-4e3c-9ab0-49d1d94eace2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:17.197070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:17.197925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:17.198695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/486] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:17.199818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe5149-fadd-488c-ab75-3a5af7fd1876 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.200058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe5149-fadd-488c-ab75-3a5af7fd1876 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.204408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe5149-fadd-488c-ab75-3a5af7fd1876 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.206169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/487] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:17.211009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe5149-fadd-488c-ab75-3a5af7fd1876 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.220398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.221097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.221380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.221657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.222403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:17.222708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.224094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e067c-b5a7-4c15-b5a0-1f707c3fd190 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.233059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 243/488] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:17.234313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.234601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.234881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.251408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:17.252788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:17.321907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 244/489] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:17.322781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.334039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe5149-fadd-488c-ab75-3a5af7fd1876 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.386012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.386681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.386965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.387241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.387984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:17.388854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.390220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.400670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/490] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:17.401494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.466851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.468248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.471382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f79ef-7636-4b46-800e-1c453a58769f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.509263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.510052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.510331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.510611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.521708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.522469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48639a-80bc-47c7-9971-319477f05da7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.523166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:17.542687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:17.543566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:17.544682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:17.544682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:17.544682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:17.544682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:17.544682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 245/491] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:17.544959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.563069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.564381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.564862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.565398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.576679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:17.584951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:17.585829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:17.586591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:17.588985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/492] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:17.590035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.665933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a70fd-9d2f-48bb-a8a2-aa22a573c9ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.695114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bf2b0-59f1-4b6d-8d79-caa99ba822e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.703257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.704281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.704817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.705172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.709132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.717001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.718402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.731414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.732097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.732417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.732695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.733405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:17.733711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.735077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50ded8-b9b1-4da5-9f74-35150fe723c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:17.739479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:17.740310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:17.741154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:17.741154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:17.741154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:17.741154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:17.743339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 246/493] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:17.744471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.749024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.752003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/494] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:17.753330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0763fe-c103-4ad2-965c-f966bf3f5cee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.753624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0763fe-c103-4ad2-965c-f966bf3f5cee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.753907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0763fe-c103-4ad2-965c-f966bf3f5cee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.759981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/495] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:17.760788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0763fe-c103-4ad2-965c-f966bf3f5cee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.854801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aea627-5564-4ae8-9a83-e786f8e8f859 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.867105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0763fe-c103-4ad2-965c-f966bf3f5cee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:17.891897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.892652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.892934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.893210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.893959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:17.894284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.896296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a615c-3bf0-47d5-b041-20fbe95e3cf6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.904292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 247/496] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:17.905052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:17.905498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.905782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.906060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.906435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:17.906930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:17.907483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:17.908457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:17.908767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:17.910141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:17.928162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/497] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:17.928390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:17.928487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:17.928571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:17.961629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.987461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:17.996014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 248/498] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:17.996863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.032413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f772dd7-cfee-40e5-bc50-f0bf3e077150 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.065306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.066084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.066371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.066649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.073950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.075397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:18.090947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:18.092558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:18.093429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/499] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:18.093972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.096855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0a481-d235-46bf-8828-a6895a615446 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.129925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.130731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.131013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.131286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.141985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.143421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1cc9ed32-b971-4d98-8ac8-e8651de42169 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:18.161549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:18.162375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:18.163182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:18.163182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:18.163182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:18.163182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:18.163182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 249/500] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:18.163882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a95d6-0c83-49d7-a121-4b0464ce9848 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.164145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a95d6-0c83-49d7-a121-4b0464ce9848 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.164445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a95d6-0c83-49d7-a121-4b0464ce9848 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.165807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 250/501] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:18.166358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a95d6-0c83-49d7-a121-4b0464ce9848 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.194061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.203514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480de33-7d77-4f08-a456-206e6acd4d71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.217422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.236387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.237180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.237516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.237750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.245381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.246805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:18.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:18.261603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:18.262518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/502] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:18.263111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.272677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a95d6-0c83-49d7-a121-4b0464ce9848 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.307305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.308131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.308446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.308747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.321309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:18.332904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:18.333931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:18.334907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:18.340343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 251/503] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:18.340343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.370921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e8819-8231-433b-b568-4840319aa99a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.404922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.405584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.405864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.406142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.406967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:18.407275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.408741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46f03e-87b6-488c-b097-3cff1f4016fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.436846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c75ef-5506-4551-8e12-a24191798a4b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.444542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/504] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:18.445754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.446048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.446174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.446325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.463421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:18.464635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:18.467331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.474217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.474877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.475162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.475439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.476157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:18.476491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.477868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.490363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 252/505] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:18.491083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.531406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/506] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:18.532262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.589181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c19aa-099b-4081-b55a-b80cd92060cf None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.625912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.626585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.626870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.627148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.627985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:18.628343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.629812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.644369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae26c3-afce-44e0-9c84-f6be30f9661f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.667483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 253/507] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:18.668348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.683000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.683669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.683963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.684281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.685039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:18.685360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.686738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.694076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.694667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/508] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:18.695449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.701574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.768618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc9d5d-74d2-46b2-9271-a2ed662a26f6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.799607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9656255-774b-41f7-9821-55b9755b4aaf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.802750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.803397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.803687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.803967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.833324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.834105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.834390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.834663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:18.842516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.843967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:18.858055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:18.858955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:18.859865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/509] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:18.860958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.875402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:18.876919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:18.902432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.920633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:18.959463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc684b-e179-4293-8026-37dc5ccfa0db None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:18.973740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 254/510] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:18.974582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:18.992076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:18.992879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:18.993160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:18.993444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.004327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.005752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:19.022165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:19.023174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:19.024251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:19.024251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:19.024251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.024251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:19.024251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/511] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:19.024251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.074886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d0a10-1389-455e-bc4d-b2383b02143a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.099799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.108592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.109241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.109531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.109817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.121846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44001-8d93-488f-b46f-b27e5f68b366 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.135698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.156334 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.157170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.157465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.157773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.168261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.169755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.177931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.179475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:19.187268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:19.188367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:19.189420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:19.189420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:19.189420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.189420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:19.189420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/512] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:19.189420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.283948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 255/513] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:19.284751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.290456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88e649-a28a-46c4-bef3-920ec24fdbf0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.310951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.325366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.326075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.326364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.326645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.327489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:19.327797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.329278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.346794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/514] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:19.347689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.375093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.389979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43baa7-0aef-4dcf-8ae3-735ebe4b8580 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.423879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.424698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.424989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.425268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.435673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.437190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.449255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752bc9c-71e1-441a-99c6-aab9d7bcee72 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:19.454231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:19.455217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:19.456645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:19.456645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:19.456645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.456645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:19.456645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 256/515] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:19.456645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.485487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.486144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.486430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.486711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.487432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:19.487739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.489128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.500854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/516] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:19.501634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.537572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.559159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413ae31-c515-4af2-803b-034cb2db28ee None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.584384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.591718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.592459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.592743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.593018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.593761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:19.594069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.595448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.600529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62de10-2eb2-4079-a996-1351c3ff2b6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.607715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 257/517] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:19.608395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.640236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.641027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.641316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.641599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.652165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.654226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:19.674898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:19.675961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:19.676907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:19.676907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:19.676907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:19.676907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:19.676907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/518] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:19.677322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.721281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15fcee-2532-4e76-bf07-6cc740f78646 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.755312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.755957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.756269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.756556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.764607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.780306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a08d5e-962a-48ad-8e6c-e14ca56733e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.813129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.813798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.814089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.814370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.815083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:19.815390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.816801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.818981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.823437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.824957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.828980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/519] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:19.829753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.922586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 258/520] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:19.923415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:19.929007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bbae-2199-44b2-b2ba-85d0e8466e7b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:19.961583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:19.961811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:19.962470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:19.962751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:19.963024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:19.964018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:19.965163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:19.985107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/521] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:18:19 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:19.985909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.034740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03809404-764c-4523-bb41-5ddb9d3a42b4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.050727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.074822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.075677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.076025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.076345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.084618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.086116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:20.103570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:20.104581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:20.105465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 259/522] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:20.106029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.112437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f7ed4-c82e-4c3d-8677-567dfd09966f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.150847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.151511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.151797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.152074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.152823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:20.153451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.155167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.168008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/523] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:20.168916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.219931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.224473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46e9d8-fd8d-4aae-8ebc-96c8dfc566eb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.261736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.262521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.262802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.263109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.270088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.271555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.276995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a2829-a31c-4300-8832-d29774f25c2d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:20.285917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:20.286985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 260/524] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:20.288258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.295479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.319897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.320739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.321022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.321298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.331973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.333417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:20.351147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:20.352054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:20.352904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:20.352904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:20.352904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:20.352904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:20.352904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/525] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:20.353571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.406662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cf1f-3f47-40ae-9ef8-2b3a3730851b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.442366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.443345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.443811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.444091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.444970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:20.445283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.446746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.461720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011607da-69dc-443b-9291-1aa334f3e96f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.465911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 261/526] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:20.466599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.484688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.505665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.506776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.507277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.507741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.517081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.519206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:20.534568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:20.535595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:20.536479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/527] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:20.537001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.569352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.575386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827441f-cdf0-4d64-9181-8028c56d0afd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.614155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.615241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.615627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.616010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.616965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:20.617384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.619331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.639629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 262/528] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:20.640281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.659834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542997e6-9551-4d4c-b9af-f984dfc83a29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.697675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.698332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.698619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.698989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.699847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:20.700037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.701949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.718294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/529] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:20.718952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.723358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.748403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247526f-d956-415f-834c-e2d60438eb73 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.785079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.785724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.786004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.786279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.807835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.824702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4241a-88cc-46e5-9836-34b473350756 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:20.860822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.861428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:20.862304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.862420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:20.862876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:20.863336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:20.864483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:20.865682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:20.867271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:20.890713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/530] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:20.891623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:20.944734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:20.972272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 263/531] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:20.972272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.000294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add54a6-a84d-420e-abe3-1cd82db588a1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.036000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.036675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.036961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.037238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.038065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=99d07f11438d4e069eab093a8b99eeab) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:21.038379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.039815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.046198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/532] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/role_assignments?user.id=99d07f11438d4e069eab093a8b99eeab&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:21.047011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.065674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:21.089415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6fb13-0883-42eb-b02b-d11e37a39321 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.127699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.128647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.129038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.129417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.137618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.139858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:21.155456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:21.156432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:21.157240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:21.161996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 264/533] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:21.163072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.181633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0f3ed-5f79-427c-b433-53286325e89c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.205368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:21.237247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.238026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.238387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.238742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.239675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=a58d153dd25943ef80eea95893118e1e) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:21.240062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.241631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.252238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/534] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/role_assignments?user.id=a58d153dd25943ef80eea95893118e1e&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:21.252238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.342835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0d673-5ba5-4127-881b-85e3904e835d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.369754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:21.405441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.406581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.407098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.407623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.424484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.426763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:21.447017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:21.447897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:21.448798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:21.448798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:21.448798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:21.448798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:21.448798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 265/535] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:21.454341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.457122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f9e01-3326-4853-8019-c671e8a7afa2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.502149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.503177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.503655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.504123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.505687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:21.506237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.507875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.521997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/536] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:21.539855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.599946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:21.620786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174c36b-41c0-42c9-9c0e-8cef471309ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.664440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2582d-8d58-46e6-9ce8-ae5f8a80d66c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.690761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.691434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.691724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.692003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.692776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:21.693091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.694571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.707476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 266/537] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:21.710794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:21.713231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:21.732912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.733734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.734120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.734402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.736261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=ed446bf868f940e48c5adb2fbcd479f5) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:21.736586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.738090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.752873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/538] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/role_assignments?user.id=ed446bf868f940e48c5adb2fbcd479f5&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:21.856360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5669-f079-42d9-900b-131cb6e835bd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:21.908539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:21.909362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:21.909647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:21.909923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:21.924413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:21.925824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:21.947545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:21.948463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:21.949281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 267/539] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:21.956534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:22.098244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a291-0c58-4205-9c70-eb16ab3027c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:22.150760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:22.151435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:22.151722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:22.152002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:22.152824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:22.153140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:22.155116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c7f9e-9f2e-4f3e-8446-c8262685441d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:22.171515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/540] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:22.177094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:22.190298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:22.319004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d47fc-2255-4661-89e4-327ae4f3083c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:22.321198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c9dd-c3bc-48e8-b98e-f2264de2859c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:22.321483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c9dd-c3bc-48e8-b98e-f2264de2859c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:22.321767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256c9dd-c3bc-48e8-b98e-f2264de2859c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:22.323134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/541] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:22.328916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:22.329209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:22.329493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:22.347023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:22.348337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:22.374284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:22.374942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:22.375229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:22.375508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:22.376370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=fd55fa1fab59477f8d747cdf3fdeb7f3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:22.376685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:22.378136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:22.390689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 268/542] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:22 2022] GET /identity/v3/role_assignments?user.id=fd55fa1fab59477f8d747cdf3fdeb7f3&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:22.429403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:22.454993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/543] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:22.492687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:22.589087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:22.654266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352ce41-aabb-4d08-9595-d9e06420a008 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:22.709991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:22.710704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:22.710989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:22.711267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:22.732267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:22.739919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:22.740836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:22.741604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 269/544] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:22.748733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:22.884751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfb35a-2e9e-4d94-8187-5d5b4a0a1242 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:22.931521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:22.932201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:22.932493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:22.932773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:22.933512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:22.933821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:22.935708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:22.944037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/545] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:22.949558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:23.136660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44540f-69df-4d6d-b2d7-ce0b10175812 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:23.165457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:23.170697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:23.171804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:23.172113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:23.172391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:23.183835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:23.185442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:23.203060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:23.203972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:23.204855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:23.204855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:23.204855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:23.204855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:23.204855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 270/546] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:23.213246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:23.272125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:23.325289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53eb3c-0bc5-4313-8636-6b665d68998c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:23.377686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:23.378356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:23.378644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:23.378928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:23.379707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:23.379968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:23.381381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:23.394393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/547] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:23.399835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:23.546420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1eea5-875c-4b47-8bec-efafc3ad781c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:23.640932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:23.641688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:23.641976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:23.642258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:23.660307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:23.660307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-06cc9735-e685-4e52-8fd0-b4606735a228 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:23.681229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:23.682247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:23.683614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:23.687846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 271/548] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:23.689245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:23.826752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467763f2-bf0c-4399-b889-0ce4bc3847ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:23.844635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:23.864615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88f8c3-aa0d-4647-9706-6669cb95caeb None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:23.864951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88f8c3-aa0d-4647-9706-6669cb95caeb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:23.865173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88f8c3-aa0d-4647-9706-6669cb95caeb None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:23.866096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:23.867029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:23.867451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 272/549] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:23.867753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:23.868277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:23.869594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:23.870171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:23.872670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:23.873540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f62fa-9e8e-494e-a333-aa07757a7173 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:23.873824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f62fa-9e8e-494e-a333-aa07757a7173 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:23.874099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f62fa-9e8e-494e-a333-aa07757a7173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:23.891401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-950f62fa-9e8e-494e-a333-aa07757a7173 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:23.892625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-950f62fa-9e8e-494e-a333-aa07757a7173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:23.893814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/550] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jul 19 12:18:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:23.905608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:23.950855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.022020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 273/551] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:24.023099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b990-f234-4a96-96c6-7c03f50a8c15 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.023388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b990-f234-4a96-96c6-7c03f50a8c15 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.023667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b990-f234-4a96-96c6-7c03f50a8c15 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.028444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 274/552] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:24.029760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c8053-a8ef-46e5-a50c-a2f3c49dc5c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.036234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.036234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.036234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.085679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:24.086997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:24.098199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.101497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.102209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.102779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.152813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.162558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 275/553] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:24.163413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.192511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:24.194343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:24.268584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d25aa-3949-4a93-a447-f774b25a6ddc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.271074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.304758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/554] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:24.305614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.309270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d4be8-3560-4eb1-a228-bcf6e27137e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.309726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d4be8-3560-4eb1-a228-bcf6e27137e3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.310013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d4be8-3560-4eb1-a228-bcf6e27137e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.310288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d4be8-3560-4eb1-a228-bcf6e27137e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.321202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d4be8-3560-4eb1-a228-bcf6e27137e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-633d4be8-3560-4eb1-a228-bcf6e27137e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:24.328960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:24.330009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:24.330813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 276/555] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:24.331942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b765b-2824-48dc-a7c2-a6c9a5af1c75 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.332044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b765b-2824-48dc-a7c2-a6c9a5af1c75 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.332349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b765b-2824-48dc-a7c2-a6c9a5af1c75 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.333701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 277/556] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:24.337692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b765b-2824-48dc-a7c2-a6c9a5af1c75 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.381129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.423325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b40a94-593a-4e45-9ee7-a8aaef4960b1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.447529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b765b-2824-48dc-a7c2-a6c9a5af1c75 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.460932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.462095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.462387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.462663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.473563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:24.481971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:24.483035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:24.483838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/557] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:24.485099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.490764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.491604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.491894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.492190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.504273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:24.505738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6f0df8e5-360e-4f96-b129-0ceee14ad317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:24.523246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:24.524839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:24.525735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:24.525735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:24.525735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:24.525735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:24.529583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 278/558] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:24.530655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.530946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.531224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.548095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:24.549440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:24.561570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.595730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d550d52-3ce4-46ea-92bd-01a1c676e73d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.624607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 279/559] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:24.625682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.633689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.646155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.647618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.647905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.648206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.649026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:24.649338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:24.651209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:24.659526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/560] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:24.660169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.738137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955316ef-bd36-4d66-be81-a916d5541e15 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.765344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18588d55-77a7-4f15-8131-546b0b9eb0a4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.777177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.777788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.778075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.778354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.779172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:24.779482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:24.780916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:24.793863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 280/561] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:24.793863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.805785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.806061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.806791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.807151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.807493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.808759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:24.809144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:24.810787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:24.823593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/562] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:24.824677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.876839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:24.906626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07ecbb-2c61-4365-ae7c-6e4dab74d77f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.930421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd875d7c-fda9-4008-87c5-ee8eeeb79480 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:24.940670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.941465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.941760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.942033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.952656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:24.960351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:24.961482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 281/563] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:24.962500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:24.966724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:24.967511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:24.967797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:24.968076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:24.978176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:24.979670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:24.998271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:24.999341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.005062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:25.005062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:25.005062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:25.005062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:25.005062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/564] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.005062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.028894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.063205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30829433-dd94-47c9-b698-1ef87f27ea05 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.095648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.096461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.096742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.097017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.098219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.101904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5abc-ed05-47c7-97d0-13940fd2ddd9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.108254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.109699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-96c8a073-912a-4b5b-964d-61e5957f3dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:25.127322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:25.128388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.129503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:25.129503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:25.129503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:25.129503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:25.129503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 282/565] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.130000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6160-93c5-4d9b-a9a9-56a8b13308ce None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.130286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6160-93c5-4d9b-a9a9-56a8b13308ce None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.130563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6160-93c5-4d9b-a9a9-56a8b13308ce None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.131936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 283/566] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:25.132516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f6160-93c5-4d9b-a9a9-56a8b13308ce None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.141154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.141943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.142228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.142519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.149239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.150697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:25.164283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:25.165351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/567] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.166238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.234526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f6160-93c5-4d9b-a9a9-56a8b13308ce None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.254624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.265951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f0059-7928-4883-a73d-edc6520a3e25 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.269281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.269945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.270233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.270513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.271262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:25.271572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.272999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6617e-6875-432b-8dc3-bd898756a3d8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.281371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 284/568] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:25.282480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.282766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.283050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.299956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.300658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.300917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:25.301016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.301209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.301927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:25.302093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:25.302240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.303596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.315160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/569] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:25.315952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.341405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.373494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 285/570] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:25.374346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.436782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabf4a6-c7b9-43af-892b-def68e20af36 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.471126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.471779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.472065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.472373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.473087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:25.473394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.474759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.481071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931eda6-794b-41c7-8705-3ee893dbfd52 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.486076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/571] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:25.486969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.495222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.517924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.518599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.518878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.519160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.519942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:25.520282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.521739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.534003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 286/572] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:25.534739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.577048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.596359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7782c-9da6-4ab1-b123-4dbf05cdd222 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.632768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.633559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.633847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.634140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.638938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7092eb-c78b-488c-9109-2ab600ae83d1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.644437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.645872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:25.662940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:25.663998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.670205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:25.670205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:25.670205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:25.670205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:25.670205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/573] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.670205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.673641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.674430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.674709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.674984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.682591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.684033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:25.698452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:25.699626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:25.700868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:25.704494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 287/574] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.705169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.738162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.770084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b4ad7-08c8-42a3-86fe-72461b0d5d76 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.801248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.805689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.806485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.806775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.807066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.816414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdf43e-a859-4190-b59d-f0904940915a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.818355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:25.826923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:25.828162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/575] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.829333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.858343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.859425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.859918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.860451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.871303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.872944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:25.887664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:25.888852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:25.889885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 288/576] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:25.890534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:25.959149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7f3bb-d056-4b1e-afe2-5264e4b9981b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.987258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:25.993689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:25.994338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:25.994621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:25.994900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:25.995728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=e2205868e3b04113b500a67eb814e58a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:25.996042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:25.996717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ba791-c218-4bac-bd02-07cc1214b4e0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:25.997591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.003170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/577] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/role_assignments?user.id=e2205868e3b04113b500a67eb814e58a&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.003740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.048104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.049279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.049641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.050312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.051161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.051635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.053183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.064834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.071694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 289/578] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.071694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.120696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c732d-f2de-46af-a336-9b3d1210eece None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.159796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.160630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.160989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.161350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.162209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.162523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.163991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.180323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55cfea-bcee-4f5d-aa7d-a38c14de4afe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.182727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/579] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.183769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.224449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.225483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.225836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.226182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.226914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.227220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.228668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.242804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 290/580] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.244028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.260503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.305209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8696558-f418-4715-97e0-82174314e5e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.320304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.345715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.346502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.346784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.347064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.347920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.348257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.349683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.355001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29555f00-681e-40d2-9590-9bdef8d47f5e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.369853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/581] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.370819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.395354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.396221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.396510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.396775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.408388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.409868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:26.432145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:26.433349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 12:18:26.434482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:26.438184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 291/582] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:26.438865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.499817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d9831-13fa-4cf5-a0f2-f89286f622e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.522922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.539809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.540929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.541344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.542020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.551392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ae308-4c17-4101-8d8e-f63e73fff14a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.552903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.554560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:26.575075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:26.576626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:26.577502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:26.577502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:26.577502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:26.577502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:26.577502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/583] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:26.578384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.594308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.595028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.595318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.595597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.603546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.610074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.610074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:26.631077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:26.632223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:26.633363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:26.633363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:26.633363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:26.633363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:26.633712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 292/584] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:26.634316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.707702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff597b-a736-4ac7-bcc7-1b097f983a21 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.741860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c69254-215f-47eb-ad2e-bca9399d6b01 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.747982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.748998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.749455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.749891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.760785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.762444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.770390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.780494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.781174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.781463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.781745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.782596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.782911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:26.783003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:26.783802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:26.784703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:26.784703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:26.784703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:26.784703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:26.784921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.789064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/585] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:26.790126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.805244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 293/586] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.806086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.878766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:26.906134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca752f7c-a357-41b0-9f18-443a21dad4b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.910490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97e0cf-6e3b-4c5c-a03b-616406ddcaed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:26.945354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.946038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.946328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.946609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.947337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.947650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.948163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:26.949050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:26.949156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.949600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:26.950100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:26.951002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:26.951495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:26.952897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:26.965502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/587] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.965502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:26.969713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 294/588] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:26.974117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.023486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.076609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d827b-5d37-4dcf-b202-2521bb30a35a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.083124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735c299-a964-4543-8436-4bbb360ca22f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.112998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.113671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.113958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.114243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.114973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:27.115287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.116788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.119642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.120068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.120527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.120811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.121088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.132781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/589] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:27.133130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.134067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.134968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:27.156229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:27.157184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:27.163431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:27.163431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:27.163431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:27.163431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:27.163431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 295/590] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:27.163431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.245618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a80c81-98ce-4aba-a502-c70bb0f2dbd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.270408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.277439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbd898-d854-4837-a8ff-7f5e6c5061c2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.281705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.282496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.282782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.283060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.289895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.291378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:27.304987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:27.305888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:27.306764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/591] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:27.307849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.318289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.319261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.319402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.319641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.328879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.330407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:27.344915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:27.345832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:27.346763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 296/592] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:27.347323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.371220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.418451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1124c-2b54-4de6-94cd-807f33372118 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.455295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33769a6b-ac3a-41f6-bde7-311d53b182f8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.455637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17051157-0a93-4716-9785-b2acac647566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.456480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17051157-0a93-4716-9785-b2acac647566 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.456889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17051157-0a93-4716-9785-b2acac647566 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.457174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17051157-0a93-4716-9785-b2acac647566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.464266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17051157-0a93-4716-9785-b2acac647566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.466372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17051157-0a93-4716-9785-b2acac647566 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-17051157-0a93-4716-9785-b2acac647566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:27.485049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:27.486161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:27.487248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:27.489167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/593] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:27.490168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17051157-0a93-4716-9785-b2acac647566 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.498081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.498751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.499121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.499399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.500153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:27.500490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.501874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.515372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 297/594] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:27.516195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.538105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.617240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17051157-0a93-4716-9785-b2acac647566 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.633486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ddb92-8ff5-4049-96a9-37b18374f139 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.637127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.657008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.657752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.657999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.658244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.659051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:27.659368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.660851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.671599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.672526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.673011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.673320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/595] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:27.673664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.674128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.674905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:27.675278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.676787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.697119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 298/596] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:27.697870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.778411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.792853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b30d-945d-4d3f-a3c6-284531b4d7bd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.805292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4f1e2-29ae-40a4-89af-7cd0a02b940d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.829849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.830589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.830898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.831152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.831938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:27.832279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.833751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.841925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:27.842744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:27.843029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:27.843305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:27.852580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/597] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:27.852580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.852946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:27.853929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:27.868020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:27.869205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:27.870069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 299/598] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:27.870561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:27.895633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:27.970536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032a8fa-86e7-4e18-a299-8cbfec3f82a1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:27.987504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b33dc-21d6-4852-9561-d76e976cb311 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.006582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.007463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.007742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.008013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/99d07f11438d4e069eab093a8b99eeab/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.022862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=99d07f11438d4e069eab093a8b99eeab, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.023796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.024353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.024623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.024907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.025183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.027114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:28.038482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.039950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.040375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/599] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:27 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/99d07f11438d4e069eab093a8b99eeab/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:28.041168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:28.069315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:28.070261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:28.071137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:28.071137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:28.071137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:28.071137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:28.073410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 300/600] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:28.074558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.150289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:28.169095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd507795-1bdd-454f-a18a-cf9efeff1a12 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.193665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d372a2-ddde-4584-9057-7647cdf3c79e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.206872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.207769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.208055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.208449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/a58d153dd25943ef80eea95893118e1e/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.225729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=a58d153dd25943ef80eea95893118e1e, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.227180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.237206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.238155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.238610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.239064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.240060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:28.244323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/601] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:28 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/a58d153dd25943ef80eea95893118e1e/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:28.244474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.244552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.244622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860b6fa-9d23-4d26-84a3-f5b941fb485b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.260895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 301/602] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:28.262856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c484f-1cb1-4d67-a30d-3c8e75ff2dc1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.263149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c484f-1cb1-4d67-a30d-3c8e75ff2dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.263706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c484f-1cb1-4d67-a30d-3c8e75ff2dc1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.265963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 302/603] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:28.267763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c484f-1cb1-4d67-a30d-3c8e75ff2dc1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.348623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:28.388443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec59ebf-a7df-45cb-ad3a-cef4def3d72b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.414750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c484f-1cb1-4d67-a30d-3c8e75ff2dc1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.436246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.436912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.437197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.437478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.438203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:28.439030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.440856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63ac6-b52f-477b-8747-638653196cc8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.443731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:28.455512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/604] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:28.457347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.457991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.458584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.460804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.461822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.462164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.462478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/ed446bf868f940e48c5adb2fbcd479f5/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.478443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:28.479747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:28.480747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=ed446bf868f940e48c5adb2fbcd479f5, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.483133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.505919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 303/605] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:28 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/ed446bf868f940e48c5adb2fbcd479f5/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:18:28.506746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.577340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/606] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:28.601991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.724963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db199190-68e1-4b9f-a0ed-3821f9e26d1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.755097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:28.766315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.767163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.767452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.767726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.773108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:28.780348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:28.789218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:28.790189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:28.791026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:28.791669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782628e-e227-4537-8711-dc56964cb643 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.791755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/607] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:28.798022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.889739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.890540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.890822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.891094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.909269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.910735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.911238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd550056-49de-4f61-b31b-18c686d37546 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:28.945102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:28.947306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:28.949522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 304/608] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:28.950717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:28.950717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:28.950717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:28.950717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:28.951300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:28.951300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:28.956265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:28.956441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:28.962991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/609] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:28.970752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:29.091500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51aa87-af8b-47c7-8789-9ee6faca8bac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:29.095692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:29.155510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:29.156172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:29.156480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:29.156752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:29.157537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:29.157844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:29.159278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:29.173664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 305/610] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:29.178677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:29.220614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:29.222407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d27b20-e0cc-4246-8980-d3c15ea2fc83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:29.297410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234956b5-30d8-47da-a6f8-af0e133195a1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:29.313375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:29.314173 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:29.314456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:29.314727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:29.337355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:29.338980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:29.362277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:29.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:29.364058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:29.364058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:29.364058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:29.364058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:29.364058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/611] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:29.370275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:29.384358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:29.386200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:29.386481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:29.386755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/fd55fa1fab59477f8d747cdf3fdeb7f3/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:29.416109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=fd55fa1fab59477f8d747cdf3fdeb7f3, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:29.417593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:29.444797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 306/612] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:29 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/fd55fa1fab59477f8d747cdf3fdeb7f3/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:29.538338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:29.542146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79926673-0141-4481-98ff-af488e9d5ce0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:29.585476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67f425-4506-416a-b603-06be01565913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:29.586141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f425-4506-416a-b603-06be01565913 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:29.586424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f425-4506-416a-b603-06be01565913 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:29.586699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67f425-4506-416a-b603-06be01565913 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:29.587420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f425-4506-416a-b603-06be01565913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:29.587727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f425-4506-416a-b603-06be01565913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:29.589173 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67f425-4506-416a-b603-06be01565913 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:29.620627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/613] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:29.626590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:29.725979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:29.769900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901051a-dbf1-4dcc-b177-7f268507b480 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:29.805482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:29.806273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:29.806554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:29.806825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:29.814658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:29.816161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:29.830785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:29.831741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:29.832705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 307/614] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:29.837580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67f425-4506-416a-b603-06be01565913 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:29.949129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67f425-4506-416a-b603-06be01565913 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:29.991758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:29.992522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:29.992804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:29.993079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:29.994235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:29.994544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:29.996060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:30.015265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/615] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jul 19 12:18:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:30.020727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:30.151365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98dcb9d-98c6-49c9-856d-a4bf1cd4f412 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:30.187230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:30.202608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:30.203341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.203624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.203904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.216913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:30.218431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:30.239747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:30.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:30.241605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:30.241605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:30.241605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:30.241605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:30.244282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 308/616] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:30.249280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:30.298029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:30.399830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2af076-94ce-4ede-8f12-457c0014f887 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:30.433921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:30.435101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.435385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.435659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.436398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:30.436706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:30.438067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c2222-1604-4686-8761-fb210f5bb8e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:30.449445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/617] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:30.455155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:30.541391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265077-3546-4bfa-a1fe-9c9d9ba3550d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.541898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265077-3546-4bfa-a1fe-9c9d9ba3550d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.541974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265077-3546-4bfa-a1fe-9c9d9ba3550d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.543548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/618] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:30.550392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.550691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.550975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.570854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:30.572217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:30.599913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ac070-7131-4e0c-a62d-1449de21bb5a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:30.666347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:30.667153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.667432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.667705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.694171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/619] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:30.696060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:30.697569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:30.728542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:30.729643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:30.730580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:30.735023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 309/620] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:30.736416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:30.738884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:30.762539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:30.798238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:30.867458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66685982-93a7-4632-afd2-d41038d24b3c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:30.874702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822d0e3-b121-4a5c-93ec-e624160a0591 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:30.908035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c87b-c015-4629-8902-ab48a1abf180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:30.908994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c87b-c015-4629-8902-ab48a1abf180 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.909395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c87b-c015-4629-8902-ab48a1abf180 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.909772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c87b-c015-4629-8902-ab48a1abf180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.916397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:30.917130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.917725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.918080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.919849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:30.920266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:30.921943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13c87b-c015-4629-8902-ab48a1abf180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:30.922325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7f13c87b-c015-4629-8902-ab48a1abf180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:30.930635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:30.931804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:30.932864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:30.936333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 310/621] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:30.936333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bd528-a85a-4105-a062-818787051615 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:30.936333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bd528-a85a-4105-a062-818787051615 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:30.936333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bd528-a85a-4105-a062-818787051615 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:30.943814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 311/623] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:30.944094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/623] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:30.945112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4bd528-a85a-4105-a062-818787051615 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:30.946421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.035701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.057495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.063407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4bd528-a85a-4105-a062-818787051615 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.080864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0456a6f-9a3c-4ca3-9232-fd3389c382ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.102727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.103407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.103699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.103984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.104896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:31.105326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:31.107330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75cf92-5969-4c71-8429-aa145dd1d03d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:31.116377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 312/624] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:31.117589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a580-c199-43f9-93bd-e298a2141e14 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.117880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a580-c199-43f9-93bd-e298a2141e14 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.118161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5a580-c199-43f9-93bd-e298a2141e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.121268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f5301-1aba-4d60-a314-08abd67ab793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.122220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5301-1aba-4d60-a314-08abd67ab793 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.122540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5301-1aba-4d60-a314-08abd67ab793 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.122825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f5301-1aba-4d60-a314-08abd67ab793 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/e2205868e3b04113b500a67eb814e58a/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.134338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e7b5a580-c199-43f9-93bd-e298a2141e14 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:31.135518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e7b5a580-c199-43f9-93bd-e298a2141e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:31.139718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5301-1aba-4d60-a314-08abd67ab793 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=e2205868e3b04113b500a67eb814e58a, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:31.141194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f5301-1aba-4d60-a314-08abd67ab793 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:31.160362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/625] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:30 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/e2205868e3b04113b500a67eb814e58a/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:31.160362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ddf0-19ce-4d0e-9a07-1bf8f8e0942a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.160362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ddf0-19ce-4d0e-9a07-1bf8f8e0942a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.160362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ddf0-19ce-4d0e-9a07-1bf8f8e0942a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.160362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/626] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:31.160362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ddf0-19ce-4d0e-9a07-1bf8f8e0942a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.217518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 313/627] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:31.218934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.219233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.219560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.242755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:31.245732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:31.328016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ddf0-19ce-4d0e-9a07-1bf8f8e0942a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.332221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 314/628] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:31.332878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.352724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.354434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.382589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.383495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.383779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.384051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.396029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:31.397561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:31.419669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:31.420675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:31.422021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:31.422021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:31.422021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:31.422021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:31.422021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/629] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:31.422021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.465157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba0d755-e394-4f60-8a0a-4a97b2e98fea None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.510167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.511995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.513051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.513550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.527677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:31.538217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:31.539269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:31.540101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:31.541726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 315/630] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:31.542320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.557280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fba5f-4d23-437e-b939-005c97850e00 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.594312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.595080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.595371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.595650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.610498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:31.618935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:31.619874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:31.621066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:31.624701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/631] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:31.625620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.651971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.667496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.683649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a1ebb-3a48-4704-93f5-55645d558be5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.728480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.729158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.729446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.729730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.730463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:31.730836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:31.733391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:31.752360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 316/632] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:31.753228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.767639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19c26d-aad4-4271-9f6e-cb7c49ed3fac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.815175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.816101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.816453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.817112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.818148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:31.818468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:31.820562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:31.831918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/633] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:31.832784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.865732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff062738-c2b1-4186-af66-e6ba4588f593 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:31.905354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:31.906178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:31.906557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:31.906931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:31.907785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:31.908218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:31.909712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:31.918896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 317/634] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:31.919812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:31.922471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.960808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:31.976152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039b0c5-6f38-4ae8-8e6e-de549070bec0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.017024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.017811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.018506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.018788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.026510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.027937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.031229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec58bfe-8cb7-4330-a859-6b1a3ccd5d54 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:32.042507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:32.043456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/635] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:32.049872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.070616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.071412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.071687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.071959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.083577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.085080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-90cdae15-1dbd-4a98-826d-97f90007df8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:32.107435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:32.108350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:32.109252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:32.109252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:32.109252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:32.109252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:32.112324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 318/636] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:32.113613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795e92-87ee-4311-8cba-8f94565b225f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.113906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795e92-87ee-4311-8cba-8f94565b225f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.114231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32795e92-87ee-4311-8cba-8f94565b225f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.115568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 319/637] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:32.116353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32795e92-87ee-4311-8cba-8f94565b225f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.156280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14108c6c-84a9-42ed-8461-3c378a86bee6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.176508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.192052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.192883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.193174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.193456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.205387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.206848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:32.223733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:32.225940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:32.225940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:32.225940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:32.225940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:32.229383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/638] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:32.230221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.232252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32795e92-87ee-4311-8cba-8f94565b225f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.234210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.271828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.272522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.272806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.273090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.273920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:32.274232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.275676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49319905-1cf6-4a67-85a8-ec8ca44a4832 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.293190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 320/639] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:32.297789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.298079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.298489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.315634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:32.316838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:32.339257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efb503-af7b-4e1c-9834-b85661f8c4c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.376015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.376646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.376913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.377194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.377913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:32.378227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.379584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.397125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/640] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:32.397125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.401509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 321/641] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:32.402489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.454445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.469742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.511781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d6dc8-7862-4056-a58e-72f62259c7c1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.518420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff8e7c-b990-4436-a24b-48434f592c9b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.548372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.549037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.549326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.549609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.550327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:32.550642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.552005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.557169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.557973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.558410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.558686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.563550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/642] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:32.564945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.631485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.633337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.675782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c4f0f-7732-4f38-bb4c-72f89c351174 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.680388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.700726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.711980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.712860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.713150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.713436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.722178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.723972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:32.738170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:32.739191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/643] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:32.740140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.755298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 322/644] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:32.755298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.850270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2001a3-766d-4995-b349-3e472281103a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.858528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f104f-1679-4b4d-be66-aa4999fda0d5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:32.885604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.886321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.886612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.886900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.895735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:32.896784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:32.897145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:32.897428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:32.897856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:32.904830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:32.905871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:32.906800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:32.907765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:32.908466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:32.908536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/645] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:32.908536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:32.927966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:32.928956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:32.929882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 323/646] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:32.930473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:32.933025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:32.964886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.025647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe51e0-ecd2-4cdc-a9be-2c7d22083b6e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.047895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf4730-bfa3-40ae-a02a-39a1d6a0cd9c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.059612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.060306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.060588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.060861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.061699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:33.062012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.063447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.081026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/647] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:33.084987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.085201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.086165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.086448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.086726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.098308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.099771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:33.117219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:33.118113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:33.119254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:33.119254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:33.119254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:33.119254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:33.119254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 324/648] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:33.119547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.178965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.198122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd888b7-13b1-4b05-a26c-f3ee9247120e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.199713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.228569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8cdd2-70cb-49ab-831d-e10ff526820a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.234865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.235767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.236373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.236652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.237394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:33.237700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.239201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.253794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/649] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:33.253794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.266563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.267246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.267538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.267824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.268712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:33.269032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.270479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.288609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 325/650] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:33.289425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.358196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ecbfb-11b0-4e66-b1fa-13f5952a857b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.388719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702c124-e43b-4e35-be1c-e17947b364ea None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.392355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43073c-9ec3-4da2-9607-e4266ab578cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.393001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43073c-9ec3-4da2-9607-e4266ab578cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.393289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43073c-9ec3-4da2-9607-e4266ab578cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.393569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43073c-9ec3-4da2-9607-e4266ab578cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.401534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.423278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.423959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.424258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.424543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.425270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:33.425578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.425956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.427005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.438882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 326/651] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:33.439619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.462914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43073c-9ec3-4da2-9607-e4266ab578cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.464382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43073c-9ec3-4da2-9607-e4266ab578cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.538815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6078b-9c4a-4df0-bc5b-2041fceec8f5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.560654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/652] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:33.561956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e9684-54f4-4798-a8b3-29a678de40cd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.562244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e9684-54f4-4798-a8b3-29a678de40cd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.562525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e9684-54f4-4798-a8b3-29a678de40cd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.563774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/653] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:33.564586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e9684-54f4-4798-a8b3-29a678de40cd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.573739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.574529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.574817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.575139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.585554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.587002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:33.604036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:33.604950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:33.605951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:33.605951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:33.605951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:33.605951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:33.605951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 327/654] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:33.606913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.630826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.631947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.670027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e9684-54f4-4798-a8b3-29a678de40cd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.705793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4a065-1b76-439a-9c90-7e9fc85b4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.706352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4a065-1b76-439a-9c90-7e9fc85b4a66 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.706635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4a065-1b76-439a-9c90-7e9fc85b4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.706912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4a065-1b76-439a-9c90-7e9fc85b4a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.708057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64046b-f6d9-47d9-b5a7-f7d5625ef8b2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.746036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.746837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.747120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.747401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.754205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.755695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:33.770202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:33.771233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:33.772125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 328/655] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:33.772638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.785382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4a065-1b76-439a-9c90-7e9fc85b4a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.786826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4a065-1b76-439a-9c90-7e9fc85b4a66 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.846329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.853877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:33.870435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca0c8-cb3c-4ef4-9e9e-4d3ca5dcf060 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:33.884229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/656] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:33.885294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.885575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.885853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.903108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:33.904305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:33.904439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:33.905231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:33.905519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:33.905800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:33.915722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:33.917195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:33.936478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:33.937497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:33.938501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:33.938501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:33.938501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:33.938501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:33.938501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 329/657] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:33.938501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:33.972378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/658] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:33.973138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.050685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e4e7d-dfca-4363-a6d3-a07eabbbb96f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.077855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.084480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.084807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ba80b-24f9-4703-9856-64630305fea4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.085143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.085428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.085706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.086422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.086727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.088099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.090869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.100831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 330/659] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.101618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.124657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.125321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.125612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.125896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.126683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.126997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.128460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.139714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/660] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.140522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.203836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650b51-e677-43b9-8af1-53ba0157c8ff None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.237090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5224f7-6e35-4f0d-bd82-9c8683a52c65 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.237896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.238683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.238967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.239247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.249907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.251389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:34.268254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:34.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:34.270412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:34.270412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:34.270412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:34.270412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:34.270412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 331/661] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:34.270412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.274663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.275370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.275655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.275962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.276702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.277008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.278872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.290358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/662] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.291133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.306938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.322273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.373149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968051b-ce5c-4da3-a52c-d98852ec9bde None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.392106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29338c-34df-44e4-8fe7-3b7875cc1e29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.406100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.406889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.407174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.407455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.417353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:34.424642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.425420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:34.425521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:34.426555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:34.427385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:34.428018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.428018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.428205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 332/663] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:34.428205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.435288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.436794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:34.452020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:34.453100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/664] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:34.454010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.528514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.534373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10cd36-b4b6-47d5-a494-dfff50162e78 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.542981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.555072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e9d0e-8fa5-436f-9ab5-55eb5d220673 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.569245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.569899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.570184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.570459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.571364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=a58d153dd25943ef80eea95893118e1e) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.571701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.573184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.579062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 333/665] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/role_assignments?user.id=a58d153dd25943ef80eea95893118e1e&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.579844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.588375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.589061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.589345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.589621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.590364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.590732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.592163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.605288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/666] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.606690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.702819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820f72-d555-44f5-8e44-71f376d0b28f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.722558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1fa47e-531c-42b4-bbd8-3ae901ed9e4f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.746783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.747598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.747884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.748163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.755684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.757404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.760817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.762266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.762266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.762266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.762780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.763096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.764841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.766083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:34.771897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:34.772829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:34.773711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 334/667] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:34.774774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.780851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/668] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.781141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.808046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:34.883395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5f293-7afe-40a5-8c17-d5645821d088 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.890488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60605d-6c85-4095-81a5-739f175ebd29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:34.919791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.920557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.920871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.921302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.922085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:34.922399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.924051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.924261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:34.925050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:34.925338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:34.925627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:34.932836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:34.934805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:34.948630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 335/669] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:34.949099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:34.950044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:34.950919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:34.951343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:34.951414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/670] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:34.951414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.012823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.052244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.063973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229dd9fa-6885-410e-a37c-371c01685b87 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.070681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf5a3c-219d-411d-9b32-54fc9357b668 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.105279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.105955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.106287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.106570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.107367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:35.107695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.109690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.110435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.111377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.111902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.112359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.127161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 336/671] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:35.127337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.127418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.127525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:35.145835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:35.146916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:35.148001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:35.148001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:35.148001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:35.148001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:35.153930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/672] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:35.153930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.238505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59aa35-d58b-4fbf-b357-2c637fb3e824 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.256686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86592c86-185d-46ac-b9d2-6ac12b6a3c56 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.267309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.274508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.275169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.275460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.275743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.295384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.297011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.297970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.298708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.299062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.299877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:35.300208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.301659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.318145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/673] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:35.318145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.348524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.350109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.432208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aafbd4-631a-4c99-987d-9083aa4ca11f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.459061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 337/674] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:35.459798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.481278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.482292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.482624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.482952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.483741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=99d07f11438d4e069eab093a8b99eeab) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:35.484061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.485542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.491499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/675] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/role_assignments?user.id=99d07f11438d4e069eab093a8b99eeab&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:35.493088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.534211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.535915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.600408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2239a9-30bc-4e26-adfa-5ef0c5a05069 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.632359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bebc44-689c-4e8f-b1bb-c5c6a5919a64 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.654405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.655318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.655745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.656194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.656934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:35.657267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.659154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.672849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 338/676] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:35.673682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.682202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.682872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.683592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.683894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.685324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=ed446bf868f940e48c5adb2fbcd479f5) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:35.685642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.687764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.694820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/677] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/role_assignments?user.id=ed446bf868f940e48c5adb2fbcd479f5&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:35.695581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.814268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e4164-92c1-4941-a595-e3376ae28dbd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.826812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.845133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:35.852622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3024a-dbb4-4a6a-b202-658715c3b947 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:35.869394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.870795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.872062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.872395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.884658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.886153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.897733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:35.898586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:35.898840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:35.899350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:35.908725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:35.909793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:35.910639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:35.911489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:35.911489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:35.911489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:35.911489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:35.911712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:35.911781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 339/678] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:35.916674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:35.928891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:35.929848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:35.930763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:35.934133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/679] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:35.939377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:36.037688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e26c55-33ae-41f0-9c07-67d47c9ba9b9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.057747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e6036-6471-434b-84d3-c3b4a537868c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.074269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.075392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.075877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.076416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.077336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:36.077861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.079443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.092404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 340/680] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:36.094301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.094977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.095269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.095552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.096571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:36.096743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.097766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:36.098260 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.121286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/681] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jul 19 12:18:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:36.126804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:36.126940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:36.145638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:36.208994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c117b1-2798-4bf2-9dba-c957ba9a278c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.237157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3f2e2-77e6-4e71-951c-ff69c1d9a9b9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.248494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.249519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.249800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.250070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.259622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.261071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.273053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.273708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.273997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.274283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:36.277192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:36.278904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:36.279754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:36.280929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 341/682] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:36.289156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:36.374438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.376602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.405957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:36.444560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:36.456303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5edb32-3e64-4c95-849d-10f73d2d7c1d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.492656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.493950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.494442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.494916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.495943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:36.496504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.498150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.511342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 342/683] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:36.524372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:36.593568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/684] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:18:36.594492 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:36.716465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828d3f1-b4b5-4f35-8812-9c0063ce676b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.778908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf57f34-59ab-447c-a203-329d63b42273 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:36.791659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.792358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.792646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.792926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.793804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=fd55fa1fab59477f8d747cdf3fdeb7f3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:36.794121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.795585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17419e1-a1ff-463b-99bd-a26ab8dc2aa2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.805704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 343/685] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:36 2022] GET /identity/v3/role_assignments?user.id=fd55fa1fab59477f8d747cdf3fdeb7f3&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:36.816555 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:36.832894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad271-fc85-4fae-b4a5-4f60ef696fc9 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.833101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad271-fc85-4fae-b4a5-4f60ef696fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.833348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad271-fc85-4fae-b4a5-4f60ef696fc9 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.835016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 344/686] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:36.842402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.842600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.842896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.856759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:36.857779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:36.858598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:36.858911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:36.859194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:36.867685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:36.869192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:36.886364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:36.888440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:36.914591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:36.915554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:36.916462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:36.916462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:36.916462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:36.916462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:36.921380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/687] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:36.929405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.018124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 345/688] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:37.048281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.084782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e02c56-e115-4922-8d06-a56377ae7bb5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.128087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.128791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.129773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.129773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.130081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:37.130394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.131770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.150573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/689] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:37.151460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.166917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5380138-7158-4cfb-8b17-331f1f8a5b4c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.183487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:37.207851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.209126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.209806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.210693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.211568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.213720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.221387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:37.247505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:37.248575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:37.254437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 346/690] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Jul 19 12:18:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:37.264863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.278807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990874b-ea2c-4b85-836f-434c2c06c306 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.317441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.318591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.319107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.319600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.328395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.330127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:37.357702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:37.358731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:37.360197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/691] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:37.368059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.487251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2716787-06c9-4ee7-b9a1-cc5d3f84294e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.528730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:37.554209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.554808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.555096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.555378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.556103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:37.556449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.557850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.560895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278773c1-e145-4cbe-a67f-7655ab272748 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.572018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 347/692] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jul 19 12:18:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:37.578117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.633983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.634653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.634940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.635229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.636024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:37.636369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.638354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.667608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/693] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:37.669990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:37.706362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:37.770851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef42f6-841c-432b-bae1-f1cba0e368c8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.825784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.826381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.826669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.826954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.827676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:37.827988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.829418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da0954-e696-468d-964e-e74aa5c13358 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.835588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 348/694] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jul 19 12:18:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:37.848458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139a897-b59e-4a51-bddf-2e15e8deb843 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:37.885572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:37.886418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:37.886858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:37.887360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:37.888351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=e2205868e3b04113b500a67eb814e58a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:37.888774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:37.890445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafe9e6-eb4b-41cf-80cd-10b5b219e721 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:37.897674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/695] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:37 2022] GET /identity/v3/role_assignments?user.id=e2205868e3b04113b500a67eb814e58a&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:38.043552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce21f6-08c9-4525-bac1-3643302353f1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.043890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce21f6-08c9-4525-bac1-3643302353f1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.044136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce21f6-08c9-4525-bac1-3643302353f1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.045808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 349/696] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:38.051685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.051985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.052319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.075027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:38.076361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:38.079653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:38.155307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/697] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:38.179050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce21f6-08c9-4525-bac1-3643302353f1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:38.289018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce21f6-08c9-4525-bac1-3643302353f1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:38.318315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:38.327095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6bafa-e3b8-4bd2-aca6-b72aeb93d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:38.328050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6bafa-e3b8-4bd2-aca6-b72aeb93d489 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.329348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6bafa-e3b8-4bd2-aca6-b72aeb93d489 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.329632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6bafa-e3b8-4bd2-aca6-b72aeb93d489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.340978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6bafa-e3b8-4bd2-aca6-b72aeb93d489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-95d6bafa-e3b8-4bd2-aca6-b72aeb93d489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:38.348848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:38.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:38.350996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 350/698] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:38.356784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:38.447612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca143e-6595-4fad-b95f-5a058bbce674 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.447931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca143e-6595-4fad-b95f-5a058bbce674 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.448168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca143e-6595-4fad-b95f-5a058bbce674 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.450377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 351/699] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:38.455478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.455774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.456062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.472817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34772c38-1aec-4165-9da5-c061c098f24b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:38.477128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:38.478340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:38.514060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:38.515191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.515701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.516201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.517235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:38.517715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:38.519193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:38.529820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/700] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:38.535195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:38.540751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:38.576714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 352/701] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:38.627064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:38.665575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bd91a-690a-49bc-906b-bb70b6e66b80 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:38.702050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:38.703208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.703702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.704231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.716652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:38.718420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:38.737158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:38.738129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:38.738998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:38.738998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:38.738998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:38.738998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:38.741306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/702] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:38.747644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:38.832896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b09dd3-baea-41a6-9080-ce5ce4f17183 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:38.855716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:38.870001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:38.913261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:38.914462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.914956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.915318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66b42a8-e302-400a-b716-63388ef2d903 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:38.915678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.938277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:38.940064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:38.950737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:38.951401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:38.951685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:38.951966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:38.952707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:38.953015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:38.954375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:38.970138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/703] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:38.975709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:38.980402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:38.981827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:38.982715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:38.982715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:38.982715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:38.982715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:38.982715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 353/704] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:38.988012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.088511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d611b5-3221-4d67-bbc7-58882a7b9a77 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:39.123916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:39.124657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.124993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.125272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.128502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:39.133247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:39.135096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:39.151421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:39.153748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:39.156001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/705] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:39.164202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.188090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67a631-74c4-4d04-957d-eae94540396f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:39.227353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:39.228031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.228369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.228656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.229399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:39.229712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:39.231668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:39.244241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 354/706] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:39.249740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.279454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea793d-7db1-4ea8-b0de-c4ac36c1521c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:39.287508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:39.352981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:39.354068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.354562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.355046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.356298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:39.356821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:39.357384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9e2b2-d43c-4a9a-8ae9-3ee5557e3a57 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:39.358743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7707312-6a21-4704-8596-c48fd8dfd15a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:39.390689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/707] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:18:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:39.392449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb84112-50e9-4507-a572-eb576ece59c9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.392958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb84112-50e9-4507-a572-eb576ece59c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.393440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb84112-50e9-4507-a572-eb576ece59c9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.394330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8527a21-a367-441f-9198-81f6b1ef1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:39.395009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527a21-a367-441f-9198-81f6b1ef1609 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.395589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527a21-a367-441f-9198-81f6b1ef1609 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.395829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/708] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:39.395941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8527a21-a367-441f-9198-81f6b1ef1609 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.396645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8527a21-a367-441f-9198-81f6b1ef1609 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:39.397069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb84112-50e9-4507-a572-eb576ece59c9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.398122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8527a21-a367-441f-9198-81f6b1ef1609 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:39.423214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 355/709] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:18:39 2022] POST /identity/v3/users => generated 254 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:39.429239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.432342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.433222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.454545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:39.487892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:39.493745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:39.641234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 356/710] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:39.667560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb84112-50e9-4507-a572-eb576ece59c9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:39.670574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.696870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:39.709209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:39.710202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.710595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.711712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.794528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aaba8-9bed-4454-9119-66e332eb6cbb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:39.801576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:39.803163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:39.835757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a658073-020e-4052-8d36-56738c343674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:39.836849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a658073-020e-4052-8d36-56738c343674 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:39.837280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a658073-020e-4052-8d36-56738c343674 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:39.837730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a658073-020e-4052-8d36-56738c343674 None admin] PATH_INFO: `/v3/services/network` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:39.838534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a658073-020e-4052-8d36-56738c343674 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:39.840083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a658073-020e-4052-8d36-56738c343674 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:39.841041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1a658073-020e-4052-8d36-56738c343674 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:39.854821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:39.856420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:39.857268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:39.860102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 357/711] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:39 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:39.867400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a658073-020e-4052-8d36-56738c343674 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.938784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/712] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 19 12:18:39.943992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:39.981825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:40.004697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a658073-020e-4052-8d36-56738c343674 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:40.068661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:40.069333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.069621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.069901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.070624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:40.070936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:40.072343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:40.086238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 358/713] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:18:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:40.093080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:40.115111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672063b7-b03f-4290-8ac5-3a38d8655b0d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:40.209488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:40.218594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:40.219390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.219678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.219962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.229942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527359b-db3f-469e-b7a2-cfc53f5c3698 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:40.241836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:40.243573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:40.281130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:40.282109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:40.283049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:40.283049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:40.283049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:40.283049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:40.283049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/714] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:40.288376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:40.289246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:40.290188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.290676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.291153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.292087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:40.292624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:40.294391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52662-6bd9-4507-b2b7-28edf9b6dee4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:40.311859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 359/715] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:18:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:40.392894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bd0d1-56d7-4dd7-8581-0fd693fd93bb None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.393250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bd0d1-56d7-4dd7-8581-0fd693fd93bb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.393469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bd0d1-56d7-4dd7-8581-0fd693fd93bb None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.394855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 360/716] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:40.417972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.419832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.420526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.428869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e60ac-c6d4-45a0-94b4-60a986e37afb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:40.454655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:40.457759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:40.462919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:40.504447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:40.505155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.505443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.505725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.506558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:40.506871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:40.508267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:40.522607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/717] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:40.546631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:40.554627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 361/718] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:40.585656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:40.635278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:40.744760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeefae-d61c-4963-ad44-b301d4ee6944 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:40.766069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e74b8-6fd4-448c-be08-2e55b58df188 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:40.809605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475d3aa-49b8-4508-8449-421470f3df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:40.810274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d3aa-49b8-4508-8449-421470f3df54 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.810558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d3aa-49b8-4508-8449-421470f3df54 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.810830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475d3aa-49b8-4508-8449-421470f3df54 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.811551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d3aa-49b8-4508-8449-421470f3df54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:40.812960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475d3aa-49b8-4508-8449-421470f3df54 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:40.830174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 362/719] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:18:40 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:40.830692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:40.871904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:40.872741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.873030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.873311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.889632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:40.891123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f2cf033f-df51-465d-b021-5f5f41413272 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:40.913058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:40.914022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:40.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/720] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:40.915421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c7627-cbcf-443b-bf4a-cae839db5c9c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.915421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c7627-cbcf-443b-bf4a-cae839db5c9c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.915421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c7627-cbcf-443b-bf4a-cae839db5c9c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.916913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 363/721] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:40.923237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c7627-cbcf-443b-bf4a-cae839db5c9c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:40.923972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:40.924313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:40.924598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:40.958022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:40.959279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:41.068978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/722] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 19 12:18:41.092735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c7627-cbcf-443b-bf4a-cae839db5c9c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:41.092944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:41.127016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:41.137322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:41.138058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:41.138349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:41.138628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:41.140098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:41.140452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:41.142433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:41.159590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 364/723] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:41.165184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:41.215537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cc44a-f3fe-41ff-97ae-3738c923042d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:41.244539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:41.259739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:41.260638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:41.261088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:41.261369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:41.275130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:41.281224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e619a-c205-47e4-a16b-0be7027465bb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:41.285140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:41.286042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:41.286850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/724] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:41.292156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:41.343933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:41.344545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:41.344832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:41.345115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:41.348396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=99d07f11438d4e069eab093a8b99eeab) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:41.348719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:41.350181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:41.363911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 365/725] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:41 2022] GET /identity/v3/role_assignments?user.id=99d07f11438d4e069eab093a8b99eeab&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:41.427577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:41.442445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5abbcc-6f7f-40ea-8251-05992590d209 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:41.496986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:41.497678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:41.497964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:41.498248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:41.499012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:41.499367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:41.500799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:41.510293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/726] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:41.515590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:41.638811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac983f3-495a-437b-adc4-a20ed0906973 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:41.663826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:41.680915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:41.681727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:41.682012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:41.682291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:41.694010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:41.695553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:41.713416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:41.714347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:41.715780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:41.715780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:41.715780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:41.715780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:41.715780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 366/727] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:41.721251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:41.892827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7604913-ca29-4e86-892a-fbc5a431dec6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:41.941262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:41.941933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:41.942223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:41.942507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:41.943231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:41.943542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:41.944936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:41.963500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/728] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:41.969081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:42.098990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ab3ed-e739-4616-8b8d-d85325f098b0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:42.134444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:42.135276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.135602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.135884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.144158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:42.145581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0ea2124f-b74a-4b98-a3f0-44bbde9ff573 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:42.164260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:42.165176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:42.166089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 367/729] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:42.171490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:42.197412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7efb-a433-4805-85ab-77421cac2422 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.197714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7efb-a433-4805-85ab-77421cac2422 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.197960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f7efb-a433-4805-85ab-77421cac2422 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.200246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 368/730] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:42.203068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:42.209037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.209465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.209768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.233968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:42.235148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:42.262373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:42.318634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487a04c-549a-483a-862f-4b87bc5fb581 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:42.334343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 369/731] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:42.370450 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:42.371597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.372091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.372627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.373675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:42.374246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:42.375888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:42.383024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:42.416331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/732] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jul 19 12:18:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:42.424113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:42.519089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d5d6-ef1e-48da-b5fb-7083163caf3a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:42.540955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:42.576032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3eda2-06b0-4671-930c-b70ae000454e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:42.603900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:42.606546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.607057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.607557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.627376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:42.635652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:42.636575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:42.637483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:42.645637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:42.646545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.647064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.647547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.656921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 370/733] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:42.658062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:42.709756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:42.764802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:42.766722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:42.858545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f390e-584e-4314-befd-aefa4ee3b692 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:42.899957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:42.915237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:42.915938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.916264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.916603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.917361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:42.917711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:42.919366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57c858-b42d-4927-a2fd-414ea8199650 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:42.927947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 371/734] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:42.929522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50322f-9097-45d4-9020-0f3ceac0887e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.929811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50322f-9097-45d4-9020-0f3ceac0887e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.930095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50322f-9097-45d4-9020-0f3ceac0887e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.935537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 372/735] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:42.936593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/736] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:42 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 19 12:18:42.937303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:42.937600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:42.937887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:42.938349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:42.960582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:42.961693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:43.040400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 373/737] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 19 12:18:43.041320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.048983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f3bd1-3dc3-44ba-abd8-4ca33ca2094b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.081150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.094399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.095201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.095491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.095772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.110504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.113069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:43.139523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:43.140504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:43.141383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:43.141383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:43.141383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:43.141383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:43.141786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/738] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:43.148264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.169833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806af6a-429e-40d6-8ba7-a5282cc6bb2d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.176777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.205934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.206852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.207201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.207548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.218902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.220430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:43.238945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:43.240861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:43.242726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:43.242726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:43.242726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:43.242726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:43.242726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 374/739] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:43.246704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.260525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1c702-30c3-41fc-9b6d-0d006c973b32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.298304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.299053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.299238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.299525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.300299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.301773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.316759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.325290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/740] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:18:43 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:43.326401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.410485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbc694-3b6d-495e-ab02-48e0de8c1c96 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.431060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.476914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec894bb2-67d4-47c0-9fa4-9202cf3ebd97 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.478621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.479320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.479711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.479976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.480805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:43.481121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.482561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2aea53-6bde-4b40-8dfa-3276b1583f95 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.496756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 375/741] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:43.498197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d8ec-9a7b-46af-b0f1-daa51dc7e694 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.498494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d8ec-9a7b-46af-b0f1-daa51dc7e694 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.498778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7d8ec-9a7b-46af-b0f1-daa51dc7e694 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.501388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 376/742] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:43.502355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7d8ec-9a7b-46af-b0f1-daa51dc7e694 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.523521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.524904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.525420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.525849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.526794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:43.527252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.530341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da8b2e-a990-44a6-9d57-44ce0f74af2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.548634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/743] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:43.550948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.551529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.551949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.570601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:43.572273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:43.642523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7d8ec-9a7b-46af-b0f1-daa51dc7e694 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.659481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/744] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:43.661002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.681663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.693244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.694200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.694585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.695036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.704842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.706536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:43.721849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:43.723033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:43.723962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:43.726347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 377/745] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:43.727404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.752655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.791353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91e7f9-ba4d-4fa6-8200-5009c00457ce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.830041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.830837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.831118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.831396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.831820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e2987-557c-4087-8167-bccd04aea465 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.839027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.841050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:43.855346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:43.856867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:43.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/746] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:43.858216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.873619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:43.874427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.874711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.875283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.875635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:43.877038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5ceb0d87-3178-448c-8026-cb9ddd41ef62 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:43.889160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:43.890096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:43.890919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 378/747] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:43.891746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f1814-af0d-450a-9745-9d985e807946 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:43.891746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f1814-af0d-450a-9745-9d985e807946 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:43.892426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f1814-af0d-450a-9745-9d985e807946 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:43.896308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 379/748] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:43.896308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f1814-af0d-450a-9745-9d985e807946 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:43.955910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:43.972137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da7f25-91ac-4572-a5e1-5cc878005323 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:43.999960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.008624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.009296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.009584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.009862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.010702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.011017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.012519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.014343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f1814-af0d-450a-9745-9d985e807946 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.030558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/749] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.031374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.059387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.059920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.060202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.060508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.061293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.061607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.063042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c27198-c7bb-41f5-a72d-7c3fac4a4540 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.075380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 380/750] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.076598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.076887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.077163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.094114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:44.095284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:44.144119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca496f-5004-4324-9fd6-3555b481b661 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.165219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 381/751] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:44.166313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.190945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.191622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.191911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.192209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.193061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.193774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.195162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.201140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/752] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.201798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.223761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.256390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.282120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b55eb3-53e2-4058-8198-30eab103fb31 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.308465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f299ff-7543-4855-811a-3fa3f78d1edc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.316046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.316719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.316997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.317268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.344925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.345690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.345978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.346263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.347090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=e2205868e3b04113b500a67eb814e58a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.347442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.349095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.360379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/753] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/role_assignments?user.id=e2205868e3b04113b500a67eb814e58a&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.361488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.400812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.403241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.469904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.505378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dedb53-05c2-482c-89b1-582e3521a300 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.524940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.549339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 382/754] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:18:44.549877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.550108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.550662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.550946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.551223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.564086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:44.571826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:44.572786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:44.573624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/755] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:44.574382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.685750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8078a-be2b-40d6-b608-e2e15de429e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.688513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee9999-0d93-48fc-aa88-b773ce037282 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.720900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.721904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.722025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.722238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.727256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.728119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.728603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.729027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.729975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.730429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.732119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.733948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.735396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.738498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 383/756] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.739500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.749303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:44.754117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:44.755191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:44.756072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:44.756072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:44.756072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:44.756072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:44.756072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/757] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:44.756388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.784469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:44.851912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c13cc1-e738-4d93-be36-e39865a69725 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.858569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666b3f8-33ce-4d23-9af7-a0d8afb51b7b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:44.887112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.888056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.888543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.889024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.889941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.890427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.892272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.893653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:44.894481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:44.894839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:44.895459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:44.896129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] RBAC: Adding query filter params (interface=public, service_id=946fdd97e0d94d1b8c788ccceedfcc95, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:44.896542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:44.897963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:44.907267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/758] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/endpoints?service_id=946fdd97e0d94d1b8c788ccceedfcc95&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.907267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.907986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 384/759] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:44.909102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:44.977385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.005356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.015029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cbc53-1ac0-4b5e-b34a-dae9adc652ec None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.016412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64249744-207d-49c6-a50e-1ba46d43edd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.051823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.053096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.053296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.053809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.053981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.054239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.054521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.054864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.055241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:45.055554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.056962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.066833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.068978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.069948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/760] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:45.070876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:45.088352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:45.089294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:45.090255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:45.090255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:45.090255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:45.090255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:45.096949 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 385/761] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:45.098075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.180043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653030c1-d65c-4e12-b964-3eb7af45dd94 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.204415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcd19-4957-4404-94ae-18c2678b8437 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.217558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.218220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.218508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.218788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.219569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.221104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.227324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.227907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/762] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:45.228997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.244762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.245950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.246962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.247476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.247857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.256483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.258416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:45.274241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:45.275201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:45.276073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:45.279282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 386/763] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:45.285041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.384407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d4c0a-e541-4d0e-aa95-560ba5f35503 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.409829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cfc6a-b9bd-4847-b1d3-99b630a1b144 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.454608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.456370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.457288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.457994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.459704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:45.460500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.462654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.473870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.474335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.474630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.474911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.475649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:45.475941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.477390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.478828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 387/764] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:45.485029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.490139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/765] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:45.498776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.546706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.589680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.630211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346180e-37be-4667-b31d-aa1f029a1954 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.640259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f4c0-988b-4d78-a8e3-08b0fa025951 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.668498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.669297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.669584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.669861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.679383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.680894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.684141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.684906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.685265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.685617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.686541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=71391e0896644c0a82a06b0905b9df24) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:45.686922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.688774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad14abe-9e0a-407d-80de-7732e4a021ee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.694981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 388/766] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/role_assignments?user.id=71391e0896644c0a82a06b0905b9df24&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:45.695803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:45.696773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:45.697892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:45.698856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f21e0-d4a9-4cb3-8405-f9a2c9a5b661 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.698856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f21e0-d4a9-4cb3-8405-f9a2c9a5b661 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.698856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f21e0-d4a9-4cb3-8405-f9a2c9a5b661 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.698856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 389/767] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:45.701355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/768] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:45.704480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.704918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.705299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.708055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.726529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:45.727946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:45.816277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 390/769] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:45.847276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:45.923609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3962687-b1b3-42b5-8a85-95cf5be5711d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:45.931429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:45.986283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:45.986966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:45.987257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:45.987540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:45.988414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:45.988731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:45.990177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:45.991946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:46.024962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/770] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:46.025858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.076090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b1656-01ba-4c5c-a333-5eaddf2c22c6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.147129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.149168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.149459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.149733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] PATH_INFO: `/v3/services/network` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.150403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.151785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:46.182179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:46.183186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:46.185482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:46.187339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 391/771] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:45 2022] GET /identity/v3/services/network => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:46.192574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.221631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b9ea5-0d32-4cb8-af6b-13269b07faad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.262602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.263399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.263681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.263956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.275119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.276592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:46.295035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:46.297186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:46.299308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:46.299308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:46.299308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:46.299308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:46.299308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/772] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:46.310115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.389671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:46.403440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:46.407111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1755d8-65a4-419d-b2d4-8bd57151abac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.470734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.471744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.472306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.472799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.473761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] RBAC: Adding query filter params (name=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:46.474266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.475835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.484349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 392/773] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:46.489394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.496640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a7c9-d74e-46c3-95b6-db35caf890a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.569992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.570658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.570941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.571225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.571944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:46.572272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.575804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.596485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/774] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:46.602216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.655825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b97ed4-d96e-4a3c-a98e-29d16f6a1e8a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.702120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.703087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.703511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.703939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.704817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] RBAC: Adding query filter params (type=network) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:46.705275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.706698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb6be9a-466f-439e-b554-67ae47cfb45b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.711946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 393/775] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:46.720297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0a564-164d-46a5-820b-cc59b6d785c6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.720587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0a564-164d-46a5-820b-cc59b6d785c6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.720919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0a564-164d-46a5-820b-cc59b6d785c6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.723113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 394/776] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:46.723674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0a564-164d-46a5-820b-cc59b6d785c6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.734205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd2d0f-37f9-4edc-b91f-7f92c6913361 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.738243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:46.756744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:46.780681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.782096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.782815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.783713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.793389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.796050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9c35bf6a-c8b2-4f87-b089-4efb215c0061 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:46.815740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:46.816691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:46.817813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:46.824724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/777] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:46.826257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.826728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.827185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.837696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0a564-164d-46a5-820b-cc59b6d785c6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:46.846232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:46.847619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:46.875571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:46.876406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:46.876900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:46.877242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:46.878116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] RBAC: Adding query filter params (interface=public, service_id=144676d10068417fb8831f726dd5106d, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:46.878503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:46.880085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:46.892272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 395/778] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/endpoints?service_id=144676d10068417fb8831f726dd5106d&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:46.893208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.924312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/779] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:46.925610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:46.994647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:47.007721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72c5ba-fb01-460a-9fb4-12b7d7d30ac5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:47.054884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:47.055673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:47.056198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.056911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.057337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.059298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:47.059784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:47.061890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:47.064239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c10217-3ef2-4e9d-ac3c-1e42cb7c7b14 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:47.079140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 396/780] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:47.081237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:47.101350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:47.102017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.102313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.102638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.103324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:47.105137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:47.116216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/781] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:18:46 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:47.116216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:47.289157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f680159-7dd6-4b89-b995-26d239a15154 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:47.326513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed931db6-beff-4154-a318-aa528ff9617e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:47.347753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:47.348957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.349530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.349972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.363621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:47.370689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:47.371874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.372294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.372670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/99d07f11438d4e069eab093a8b99eeab/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:47.374360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:47.375385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:47.376230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:47.376867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=99d07f11438d4e069eab093a8b99eeab, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:47.377570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:47.379119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 397/782] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:47.385432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:47.397138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/783] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:47 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/99d07f11438d4e069eab093a8b99eeab/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:47.421948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:47.507562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e45ac-1a4f-473c-bc0c-0b01728b4d58 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:47.543807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:47.544411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.544690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.544968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.545728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:47.546036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:47.547428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1faf3d-a7ad-4f36-b96c-440ee42d94b6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:47.556812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 398/784] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:47.563309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:47.620213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:47.770680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7348bfd-565d-4ed7-849b-aa92c8a56cf1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:47.831536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:47.833387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.834170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.834936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.844860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f841a-9bb7-42ce-bd8b-13be0caf027e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.845246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f841a-9bb7-42ce-bd8b-13be0caf027e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.845433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f841a-9bb7-42ce-bd8b-13be0caf027e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.847024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 399/785] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:47.852577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:47.852577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:47.852880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:47.857035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:47.859971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:47.872918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:47.874410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:47.894876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:47.896637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:47.898420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:47.898420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:47.898420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:47.898420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:47.902288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/786] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:47.911550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:47.990862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 400/787] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:18:48.006090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.045483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.047839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.051089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf5a59-10b2-48b5-a79d-79dd0d6d5412 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.088453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.089044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.089333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.089612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.090431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:48.090742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.092146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.104689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/788] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:48.111074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.217400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7d1d6-3148-4fe1-acb0-4cf0f4be6495 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.240724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70893801-2ca6-49ac-aa6d-2f5199841486 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.254120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.254923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.255211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.255490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.263706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.265431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.268539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.283589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.288207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.288207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.288207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.288207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0edc438d-1cf7-4c90-9dac-02da4bc294b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:48.288568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:48.289415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:48.290340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:48.290882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.290952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/789] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:48.290952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da1ea-337a-4bc5-9cc4-1059deb9e02b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.290952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da1ea-337a-4bc5-9cc4-1059deb9e02b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.290952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26da1ea-337a-4bc5-9cc4-1059deb9e02b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.291468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/790] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:48.296548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26da1ea-337a-4bc5-9cc4-1059deb9e02b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-81d9eca2-ffa3-4976-b4e4-44cf2db85984 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:48.310826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:48.311775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 19 12:18:48.312771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:48.315445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 401/791] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:48.317005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.317509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.318008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.336932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:48.338559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:48.403733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.413390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 402/792] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:48.414474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.428600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26da1ea-337a-4bc5-9cc4-1059deb9e02b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.474825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.475501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.476306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.476623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.477492 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:48.477820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.479233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.498177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/793] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:48.498939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.548170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e9efb-cf91-4070-9848-4f84b69f5fc3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.552845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.590279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.591617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.592521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.593004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.593929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:48.594437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.596887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.604436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 403/794] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:48.606191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.613637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1f57d-8d24-4af0-b6f1-9272a7932ebf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.657022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.657820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.658104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.658381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.670240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:48.678432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:48.679340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:48.680160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/795] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:48.680865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.682040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.732831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ea8b5-1553-4215-b3e3-98eac93de871 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.769406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.770265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.770556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.770834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.782227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.783857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.788452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94006b1-81a4-4daf-a167-7b7eb5d7543e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.798175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:48.804023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:48.805137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:48.806178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:48.806178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:48.806178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:48.806178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:48.808091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 404/796] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:48.809228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.831210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.831973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.832323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.832611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.833310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:48.833618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.834976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.841884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/797] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:48.842791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.917058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516dbc5-e8e7-47aa-8960-9aaec679fecc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.932736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:48.955936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9817-fcf9-4f7e-bb19-0171889515ba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:48.957311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.958214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.958678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.959134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.960058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:48.960588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.962197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:48.970963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 405/798] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:48.972098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:48.992863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:48.993775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:48.994059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:48.994335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:48.995051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:48.995355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:48.997096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.014601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/799] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:49.015368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.048654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:49.079677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768479a1-1aa7-4192-92aa-11a88c0003e8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.115118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.115800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.116205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.116540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.117470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.119093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.127818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7696a-9870-4292-9e5c-e1f08a5ea7fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.144208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 406/800] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:18:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:49.145362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.172418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.173457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.173995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.174417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.182059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:49.188465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.191816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:49.213043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:49.214000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:49.215427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:49.215427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:49.215427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:49.215427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:49.217847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/801] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:49.225280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.273758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc406b5-7000-4bc0-bd4f-819763aa263a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.366116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.367667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.367959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.368639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.377150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.378988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.383941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:49.397410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:49.398540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:49.399730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 407/802] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:49.404879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.413211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9c70d-906b-4f33-b2b0-7d59f5ce8c6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.455143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.455968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.456404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.456792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.457593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:49.458027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.459400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.483081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/803] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:49.488677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.519948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:49.531250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaee93-0f46-44dd-b7b1-a31233878c24 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.577922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.578595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.578880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.579161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.581133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:49.581452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.584331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce390af3-324d-4f3f-945f-c9bd552859eb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.615645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 408/804] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:49.621299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e878-f402-409f-8d3d-1b7693e7d682 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.622313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e878-f402-409f-8d3d-1b7693e7d682 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.622607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e878-f402-409f-8d3d-1b7693e7d682 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.624095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 409/805] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:49.626107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0e878-f402-409f-8d3d-1b7693e7d682 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.679252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc516d-2dd1-4d38-ad1b-ec39635ea435 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.717146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.718071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.718365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.718649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.727111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.728726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.731680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:49.743358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0e878-f402-409f-8d3d-1b7693e7d682 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9c2be5f9-18fe-495c-a21a-186338fd6a63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:49.746966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:49.748171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/806] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:49.749446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.750121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.750121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.767316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:49.768727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:49.783779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:49.785035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:49.785462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:49.785880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/e2205868e3b04113b500a67eb814e58a/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:49.788236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=e2205868e3b04113b500a67eb814e58a, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:49.789810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:49.806775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 410/807] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:49 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/e2205868e3b04113b500a67eb814e58a/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:18:49.808892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.828085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:49.863268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/808] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:49.947572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:49.969189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c75b6a-ed1e-4e2b-a39d-d6d6d45d93ee None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.010988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.011765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.012041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.012362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.013493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:50.014235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.015997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.040298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 411/809] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:50.046233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.099007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbb6a6-4e9e-4a0c-97cb-49d121f4eff0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.125427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.146622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.154842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.155886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.156215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.156487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.157631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.159576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.163223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334afd-577d-4604-9548-9da12e78c5ca None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e83307e5-1fe5-4859-aa5b-c4edcfadde02 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:50.178010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:50.179224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:50.180436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/810] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:49 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:50.185184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014d69-b21e-4b74-88bd-2beacc84ebf4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.185649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014d69-b21e-4b74-88bd-2beacc84ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.185931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014d69-b21e-4b74-88bd-2beacc84ebf4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.187368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/811] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:50.189028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014d69-b21e-4b74-88bd-2beacc84ebf4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.214078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.216239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.217030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.217547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.231779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.233556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f81c9135-0448-49bf-bec2-80f95bd27b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:50.257700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:50.258635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:50.259510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:50.259510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:50.259510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:50.259510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:50.262238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 412/812] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:50.263843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.264308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.264754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.286154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:50.287479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:50.313699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014d69-b21e-4b74-88bd-2beacc84ebf4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.360019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 413/813] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:50.361137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.361925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.362790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.363224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.363946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.365756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:50.366239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.367775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.375264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/814] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:50.380903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.414746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.445262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.481590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3fe0b-49d8-4925-9772-1fb0d3e467d5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.499073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e108dee-2628-49cf-8120-867bbd69807a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.518717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.519530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.519912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.520316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.521169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:50.521579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.523317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.537215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 414/815] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:50.538289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.540535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.541497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.542036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.542493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.543518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:50.544015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.545591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.556170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/816] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:50.557391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.644083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de347ff1-5e9f-48ea-bfb0-fab00b2995fd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.656274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.676386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990f7be-98a9-4fff-915e-e4ee594ea3df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.682299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.683202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.683597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.684235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.696249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:50.704459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:50.705984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 415/817] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:50.707258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.716155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.730875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.731998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.732506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.733018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.740718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.742592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:50.762415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:50.763318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:50.764214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:50.767481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/818] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:50.768728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.817272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf50f1f-9b63-4ac0-89d2-45fda7cf8967 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.865928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.866895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.867616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.868407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.870084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.872851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.889443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945fe467-1c49-4dd4-88c7-dd94b5e6eea9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:50.908613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 416/819] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:18:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:50.908613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:50.964128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:50.965082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:50.966364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:50.966644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:50.966919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:50.967753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:50.968255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:50.972090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:50.993913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/820] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:50.998198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.078256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:51.126057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50375e4-7faf-46a7-9ac7-60f2e1aab0da None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.172542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071547a-0a57-4c31-8b48-2fdaae769ba8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.176392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.177310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.177827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.178309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.179310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:51.180432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.182105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.198293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 417/821] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:51.205002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.228413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.229211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.229494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.229771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.247771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.251416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:51.275467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:51.276456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:51.277882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:51.277882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:51.277882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:51.277882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:51.277882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/822] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:51.285301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.333597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:51.350833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a94c2b-14aa-4d5f-b9c3-08529189b474 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.393865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.394758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.395039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.395375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/71391e0896644c0a82a06b0905b9df24/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.405585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:51.411077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=71391e0896644c0a82a06b0905b9df24, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.412566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.417666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53fc3d-d15b-48a9-ad86-6553b8ff95aa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.431426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 418/823] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:18:51 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/71391e0896644c0a82a06b0905b9df24/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:18:51.459689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.460901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.461394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.461889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.462845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:51.463370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.465207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a9a4e-1002-4da3-8ec1-03153a8ee181 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.480978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/824] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:51.489314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.498074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75129-637b-46b3-9acc-6548427221f2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.498263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75129-637b-46b3-9acc-6548427221f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.498550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b75129-637b-46b3-9acc-6548427221f2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.504152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/825] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:51.505671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.505955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.506230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.525731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:51.527495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:51.606124 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/826] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:51.610756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0bc48-3053-40b6-8896-942c5b7bbebe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.628729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.648086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.648919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.649207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.649489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.652438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:51.657962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.659236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:51.676440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:51.677603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:18:51.678758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:51.685460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 419/827] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:18:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:51.686982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.738306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:51.760981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ceace6-ab15-4722-a423-ccfe5a7edde2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.795749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f95220-6a26-4294-b8c5-acaf8d26fbb3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.800267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.801285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.801779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.802235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.803074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.804669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:51.817865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:51.818872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:18:51.819724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:51.822746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/828] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:51.828876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.835409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.836089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.836406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.836687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:51.837535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:51.837850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:51.839361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:51.863944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 420/829] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:18:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:51.864994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:51.957018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97713a3-b8a1-4cdd-bc39-56e777458581 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:51.996929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:51.998074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:51.998590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:51.999083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.000030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:52.000627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.002241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.008382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/830] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:52.020267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.034063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.042285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.102167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff7063-a76b-4873-874f-486ae8757032 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.133132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3be76-d95e-4e0e-94c0-95898c8fe0fd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.144481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.145363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.145807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.146323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.169705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.170412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.170693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.170969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.171669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:52.171975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.173382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c365e1-ac03-4c79-b6b3-9521124befcf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.178201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/831] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:52.179253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1c19-465a-4e97-8ff1-5ccce050eb7e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.179632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1c19-465a-4e97-8ff1-5ccce050eb7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.179918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a1c19-465a-4e97-8ff1-5ccce050eb7e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.181337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/832] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:52.188436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.188436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.188436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.206803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:52.208060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:52.227867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.229858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.301987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.302867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/833] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:18:52.331416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.335681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.368265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 421/834] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:51 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 12:18:52.371765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.498938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a91e9-5503-4cde-859e-57030bfdbabc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.541092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.542475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.543292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.543567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.557939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.560164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.567070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03b72e-2b2f-4ff1-9533-9f0d333b1342 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:52.583050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:52.584198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:52.585330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:52.585330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:52.585330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:52.585330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:52.585330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 422/835] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:52.591128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.608512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.616275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.617090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.617375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.617653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.632359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:52.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:52.644287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:52.645162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/836] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:52.650282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.728570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.741497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5cd2a-7a53-434c-8f08-7ddc9c9cdd00 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.755930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082df01-fb05-4d6b-9b5e-3ff6ecd1537d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.789078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.790080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.790539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.791004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.791394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.791947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:52.792065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.792394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.792550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.792758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.793547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:52.793856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.794330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.795212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.808762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/837] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:52.809585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.810018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 423/838] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:52.815018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:52.891238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.926315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b63796-717d-4507-9d01-19ae6b0f4170 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.927980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ccfc7-0bca-4774-8418-18f9923d7529 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:52.957384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:52.961870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.962742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.963139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.963539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.969216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:52.970174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:52.970600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:52.971031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:52.972157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.973702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.983414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:52.985062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:52.988572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:52.989650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:52.990759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:52.991906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 424/839] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:52.997620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:53.006649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:53.007616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:53.008531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:53.008531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:53.008531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:53.008531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:53.015316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/840] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:53.016608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.100030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1aaab-f917-41bf-bcc0-86a5e6fdf687 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.125959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491bf02-8f27-4a47-8572-8436302087fd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.126568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:53.137179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.137911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.138273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.138629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.139506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:53.139916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.141432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.158907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 425/841] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:53.164433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.171154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.172078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.172613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.173089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.174452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:53.174983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.176916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.197901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/842] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:53.199616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.254791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:53.278158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc909f-d6e1-4404-be2c-cf5feaf8ce7f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.323504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.325435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.325994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.326472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.327508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7e673-71ee-4dbf-817e-c38c193fe07c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.327732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=99d07f11438d4e069eab093a8b99eeab) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:53.328247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.331045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.339652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 426/843] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/role_assignments?user.id=99d07f11438d4e069eab093a8b99eeab&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:53.380691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.381913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.382484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.382957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.392379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.394215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-86d9ebf4-9325-44c8-9a6a-19e516ff2398 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:53.411304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:53.412364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 19 12:18:53.413323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:53.420323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/844] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:53.423188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.481146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:53.492006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431995d-c122-46c6-9fce-35a82fdacbac None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.492350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431995d-c122-46c6-9fce-35a82fdacbac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.492584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431995d-c122-46c6-9fce-35a82fdacbac None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.494833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/845] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:53.501747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.502183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.502554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.524673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:53.526288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:53.541869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e17b1b-7437-44a0-8b1c-ff4051953a8d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.585445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.586128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.586413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.586686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.587525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:53.587839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.589432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.607980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/846] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:18:53.620222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 427/847] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:53.629245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.631912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.666313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:53.737529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08ed2e-e43c-45bc-a0bb-ba5a64543781 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.752691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:53.764429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1390ef-4049-40ce-83c7-f46045adfe8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.781288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.782411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.782772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.783053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.799935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:53.802916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.803696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.804076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.804475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:53.808969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:53.809877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:53.811630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 428/848] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:53.815915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.880015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.881901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.910464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:53.920396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55218681-adcc-48b4-a7c3-7fd3e5caad5d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:53.955851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:53.958186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:53.958186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:53.958470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:53.959221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:53.959528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:53.960934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:53.973997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 429/849] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:53.975965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:53.996827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/850] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:18:54.006375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.008623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.083397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38954dbb-614e-4d65-9319-aed7423b8919 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.109116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947d9de-8785-4b84-92a8-57d7dc7e90fd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.118807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.119598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.119926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.120221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.131503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.132995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.138647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.148219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.149277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.149703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.150121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:54.151680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:54.152862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:54.154197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:54.154197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:54.154197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:54.154197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:54.154197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 430/851] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:54.158570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.164287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.165973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:54.185853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:54.186763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:54.187654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:54.187654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:54.187654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:54.187654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:54.194177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/852] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:54.195565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.250843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.270573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081fd22-5969-4d88-a4c1-9f3d1c54ece9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.298217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612170-4abf-4851-8a39-9f895b07008d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.306995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.307663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.307945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.308804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.309633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:54.309950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.311426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.328930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 431/853] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:54.329830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.336411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.337430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.337933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.338416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.339851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:54.340388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.341950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.358899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/854] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:54.360556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.406484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.437156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc005d52-6462-4c33-a659-5074345fc809 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.467311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe43bd-532f-450f-a251-936018193d3e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.473308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.474519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.474808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.475087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.484759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.486208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.500763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.501543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.501824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.502098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4a03f5c1-2782-48e0-86c6-8c2f4a32bdb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:54.505033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:54.505904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:54.506804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:54.508325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 432/855] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:54.509635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfbd66-619d-4ea7-94ec-724fde9790be None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.509923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfbd66-619d-4ea7-94ec-724fde9790be None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.510256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfbd66-619d-4ea7-94ec-724fde9790be None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.511853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 433/856] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:54.512132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.512675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfbd66-619d-4ea7-94ec-724fde9790be None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.514021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.517804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1ba2bca3-309c-4902-97e3-726cd2f68065 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:54.529422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:54.530351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:54.531238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/857] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:54.532160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.532452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.532688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.549128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:54.550379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:54.632507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfbd66-619d-4ea7-94ec-724fde9790be None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.633591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/858] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:54.641378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.670831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.671510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.671796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.672075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.672948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:54.673417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.674701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.675696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.699542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 434/859] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:54.699542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.754950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0929a0-d43e-40f2-a5a3-853a75264b29 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.756071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.790699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.791356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.791644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.791925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.793287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:54.793601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.795023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.803612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe95db7-9cf0-4ac7-91a5-7a275ebbe675 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.806674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/860] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:54.807230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.844243 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.844901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.845287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.845568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.846279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:54.847760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:54.868234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 435/861] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:18:54 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:18:54.868234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:54.913069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:54.925031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3e741-1837-4d5c-bb35-c563fce35a7c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:54.982276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:54.982941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:54.983230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:54.983506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:54.990799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0cbdb-348d-4a3b-8670-059547197337 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:55.014828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:55.039985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:55.041101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.041631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.042125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.043865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=e2205868e3b04113b500a67eb814e58a) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:55.044434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.046944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.055084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 436/862] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/role_assignments?user.id=e2205868e3b04113b500a67eb814e58a&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:55.075893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.077497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.174423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:55.197350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/863] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:54 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:18:55.202871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.311349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb5c94-56f1-4b0c-be94-5ea99a93cad1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:55.346852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:55.347652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.347938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.348248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.359891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.361377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e0ee23a7-18dc-45b9-90a3-675b65dc1d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:55.379476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:55.380418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:55.381302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:55.381302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:55.381302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:55.381302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:55.381302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 437/864] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:55.387206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.490603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86f393-3c1e-4d9a-ba52-e7dbd6ef44ef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:55.529288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:55.529952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.530238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.530518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.531236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:55.531544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.533102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a8b6-d0c1-4763-860d-db691e68d000 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.533523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.533616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a8b6-d0c1-4763-860d-db691e68d000 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.533701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042a8b6-d0c1-4763-860d-db691e68d000 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.535302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 438/865] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:55.540220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.540518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.540799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.545700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/866] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:55.552498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.562963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:55.564077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:55.599292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:55.629128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:55.654341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 439/867] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:55 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:55.677298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.690928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a901525-7057-468e-a0fc-1680479446f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:55.733214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:55.734103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.734428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.734708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.742326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.744195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:18:55.760065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:55.761284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:55.763010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/868] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:55.766932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.796078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5650-30a2-4895-8049-184b79c42e6b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:55.831336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:55.832443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.832925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.833383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.834231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.835797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:55.848802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:55.849721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:18:55.850532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:55.853357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 440/869] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:55.865156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.878439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f1778-7410-40e8-aeb6-8df8ac313a7a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:55.886846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:55.905634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:55.926696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:55.927356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:55.927885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:55.928164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:55.929347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:55.929662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:55.931097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:55.947734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/870] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:55.957299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:55.986906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709cc27-97cf-4221-bbdc-de94bdec67ab None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:56.030783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:56.031677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:56.032112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:56.032591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:56.033579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:56.034511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:56.036364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:56.042604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 441/871] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:56.047984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:56.084725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003231b4-ffbe-48eb-98c0-0751a4580142 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:56.125442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:56.126439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:56.126947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:56.127449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:56.128592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=71391e0896644c0a82a06b0905b9df24) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:56.129120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:56.130754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2af6c-8f68-4b56-b90b-60ad61def6e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:56.138076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/872] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:18:55 2022] GET /identity/v3/role_assignments?user.id=71391e0896644c0a82a06b0905b9df24&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:56.153032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:56.157149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f0755-3dab-4ec0-8dc0-719ab280c20d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:56.196218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:56.196923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:56.197203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:56.197473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:56.198214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:56.198521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:56.200511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf84ad-1362-4d18-b638-fac8a0c65131 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:56.212388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 442/873] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:18:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:57.114951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e8000-2057-4b37-a5f1-2d6052096b19 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.115969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e8000-2057-4b37-a5f1-2d6052096b19 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.116532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e8000-2057-4b37-a5f1-2d6052096b19 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.118606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/874] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:57.124001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.124849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.125330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.142624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:57.144055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:57.157889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694dc79-b757-4eb7-a799-1c35e2083f9a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.158166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694dc79-b757-4eb7-a799-1c35e2083f9a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.158468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694dc79-b757-4eb7-a799-1c35e2083f9a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.159939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/875] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:18:57.165215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.165763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.166263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.185289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:57.186814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:57.213433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:57.228770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 443/876] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:57.256917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.271724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/877] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:18:57.296237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.363656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:57.378264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a3e93-b2cc-46a2-8300-89423402d487 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.418307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.419238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.419591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.419938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.420708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.422164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.432115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af9e773-52b3-4d60-b1bf-76f2bfd3488d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:57.434655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:57.435814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:18:57.436874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:57.438261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 444/878] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:18:57.443493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.473635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.474446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.474734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.475015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.475768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.477393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:57.490106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:57.492376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:18:57.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:57.498465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/879] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:57.498465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.525223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:57.549806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046f9f0-c81d-4df2-b4c7-4bd2b39dff1e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.588350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.589019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.589831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.590114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.590822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:57.591130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.592762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.602320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 445/880] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:57.603302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.623438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5b0cb-7aa8-41b2-a28c-6a5c5bc23b70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.628505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:57.662048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.663212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.663701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.664205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.665133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:57.665711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.667808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.678073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/881] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:57.679297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.723754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9dd4e-d49f-40fb-9116-e8fa3fd1a889 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.761678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.762349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.762633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.762905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.763621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:57.763929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.765462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.775642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 446/882] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:18:57.776779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:57.785391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:57.799161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3668e8-52bd-4c04-9b00-56683396167a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.835740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.836543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.836901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.837254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.838376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:57.838488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.840578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.852650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/883] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:57.872261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:57.895306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595a115-808d-41f0-a115-1790a9db45fd None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:57.934411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:57.935094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:57.935383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:57.935666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:57.936513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] RBAC: Adding query filter params (interface=public, service_id=1a76c561eafd492a8e2044c6e288fb30, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:57.936832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:57.938256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa446c67-5324-4925-a757-7a68335be040 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:57.944846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 447/884] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/endpoints?service_id=1a76c561eafd492a8e2044c6e288fb30&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:57.953658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:58.064133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aeb7c4-7b92-41bf-9a85-d450fbd8c41b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:58.099794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeccaa0-0eb6-4ede-8507-7e40cf4c4a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:58.100852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeccaa0-0eb6-4ede-8507-7e40cf4c4a42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.101342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeccaa0-0eb6-4ede-8507-7e40cf4c4a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.101835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeccaa0-0eb6-4ede-8507-7e40cf4c4a42 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.102794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeccaa0-0eb6-4ede-8507-7e40cf4c4a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:58.104389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeccaa0-0eb6-4ede-8507-7e40cf4c4a42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:58.110619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/885] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:18:57 2022] GET /identity/v3/services => generated 1011 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:18:58.290096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122f57-b610-4f3b-ac43-ebfe32057d72 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.290631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122f57-b610-4f3b-ac43-ebfe32057d72 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.290631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62122f57-b610-4f3b-ac43-ebfe32057d72 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.292322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 448/886] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:58.298382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.298830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.299008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.299388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c38a9-d0ab-4828-885b-787e72a73ceb None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.299687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c38a9-d0ab-4828-885b-787e72a73ceb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.299970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c38a9-d0ab-4828-885b-787e72a73ceb None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.301345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 449/887] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:58.306420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.306712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.306995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.318067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:58.319522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:58.324542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:58.325790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:58.332122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:58.391114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/888] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:18:58.399898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 450/889] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:18:58.414853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:58.424392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:58.523080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4847c00-31fa-4743-acc2-b6f7627457a7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:58.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:58.553282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9168113-4ae1-4583-9f5d-7a63c5e2ab82 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:58.564672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:58.565983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.566347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.566706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.580399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:18:58.592098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:58.593139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:18:58.593976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:18:58.595374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:58.596314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.596774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.597216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.598422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:58.598558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/890] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jul 19 12:18:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:58.599453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:58.600596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:58.616133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:58.630136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 451/891] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:18:58 2022] POST /identity/v3/services => generated 269 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:18:58.724300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d5fd-c8de-4896-af0e-2fa941e4b175 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:58.768781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:58.769449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.769733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.770007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.770760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:58.771066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:58.772486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:58.782386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/892] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:18:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:58.788632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:58.899150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb49f9c-e800-46ac-b5ac-98ec4a7b3ba1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:58.935291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:58.936499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:58.936992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:58.937489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:58.939788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:58.950145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:58.951732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:18:58.972062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:58.973021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:58.973887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:18:58.973887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:18:58.973887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:18:58.973887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:58.973887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 452/893] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:58.981332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:59.048169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:59.087686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadf635-9e32-4c4e-b632-94d09bb69b04 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:59.123320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:59.123975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.124293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.124572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.125284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:59.125586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:59.127230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:59.140782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/894] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:59.146917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:59.253288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3a7f2-4d31-4eea-80c7-4ad7c295f24d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:59.288723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:59.289529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.289809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.290081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.297764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:59.299210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:18:59.314060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:18:59.315054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:18:59.315979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 453/895] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:18:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:18:59.321341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:59.428547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5232d-7c6c-451d-8b9a-af013bfb00e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:59.456541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:59.468048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:59.468914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.469204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.469478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.470319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:18:59.470628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:59.472053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:59.492247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/896] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jul 19 12:18:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:18:59.501247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:59.564645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:59.618591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ae625-138b-43f3-986b-00971c5d51f1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:59.653849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720577cf-4c6e-440d-ac0f-a31b931d33c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:59.654825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720577cf-4c6e-440d-ac0f-a31b931d33c7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.655320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720577cf-4c6e-440d-ac0f-a31b931d33c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.655826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720577cf-4c6e-440d-ac0f-a31b931d33c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.725868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720577cf-4c6e-440d-ac0f-a31b931d33c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:59.727379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720577cf-4c6e-440d-ac0f-a31b931d33c7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:18:59.785307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:18:59.829255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 454/897] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:18:59 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:18:59.834894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:18:59.943047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6fa2-510c-4d62-8089-560fae628258 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:18:59.959366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204371-ded5-4187-ad61-82e8c7de5ef6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.959654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204371-ded5-4187-ad61-82e8c7de5ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.959938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204371-ded5-4187-ad61-82e8c7de5ef6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.961555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 455/898] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:18:59.966558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.966849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.967129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.978867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:18:59.980619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:18:59.980906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:18:59.981182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:18:59.985474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:18:59.986641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:18:59.993227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:18:59.995059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:00.019113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:00.020023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:00.020939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:00.020939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:00.020939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:00.020939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:00.020939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/899] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:00.027739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.060338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 456/900] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:18:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:00.071356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:00.082475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.155341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835e202-133c-474f-b6e2-b6ad589ad0d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.159759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:00.186636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f03092-8981-446c-a929-1bfecaa68345 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.199580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.200297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.200574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.200861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.201586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:00.201896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.204080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.220831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/901] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:00.225719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.228955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4f7f2-c294-4172-b953-1f193cdd8d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.230160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f7f2-c294-4172-b953-1f193cdd8d12 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.230480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f7f2-c294-4172-b953-1f193cdd8d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.230739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4f7f2-c294-4172-b953-1f193cdd8d12 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.231479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4f7f2-c294-4172-b953-1f193cdd8d12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.233018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4f7f2-c294-4172-b953-1f193cdd8d12 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.260491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 457/902] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:19:00 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:00.346983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:00.355480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61209d76-c891-4d35-9642-86e11975f9fc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.373401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5656ad4-7173-42b3-bdcd-6a5fb39a3c45 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.374388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5656ad4-7173-42b3-bdcd-6a5fb39a3c45 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.374891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5656ad4-7173-42b3-bdcd-6a5fb39a3c45 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.376901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 458/903] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:00.384224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.384224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.384224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.401379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.403491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.403830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.404309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.409953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:00.410352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:00.411995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.413795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:00.432622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:00.433543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:00.434444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/904] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:00.440083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.494597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 459/905] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:00.518188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.554911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0863af8-193f-40d1-9a38-8a8827215c83 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.599778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:00.613835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.614890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.615185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.618508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.619489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:00.619827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.621763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.642337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/906] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:00.642966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84922ee-a025-4a87-8c82-d22825e1f1d4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.645328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.658942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:00.685593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.687211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.687498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.687776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.688489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.689924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:00.705892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:00.706932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:00.708156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 460/907] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:00.713185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.769331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d0b0c-21d8-430f-8685-d1994e40eb5b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.812336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.813281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.813574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.813856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.815006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] RBAC: Adding query filter params (role.id=f79951bd70da4fb6b183f9f4be427b56, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=71391e0896644c0a82a06b0905b9df24) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:00.815825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.817602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b59ef6-fb62-48bf-9e2a-ebfcc7df7a32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.824504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/908] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/role_assignments?user.id=71391e0896644c0a82a06b0905b9df24&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f79951bd70da4fb6b183f9f4be427b56 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:00.826219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f165868-c6b3-4894-8309-d60bf4da3ae8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.826520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f165868-c6b3-4894-8309-d60bf4da3ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.827152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f165868-c6b3-4894-8309-d60bf4da3ae8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.829208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/909] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:00.834840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.835149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.835410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.840741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfc152-c3d1-4b15-bec0-ba3970f32db6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:00.854274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:00.856229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:00.875228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:00.882393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:00.883338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:00.883774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:00.884363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:00.885257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:00.885737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:00.887360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:00.893608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 461/910] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:00.899615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.948849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/911] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:00.964300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:00.967816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:01.016501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c79b44-1459-4e9e-af66-34f9e45df3d2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.050735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.051554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.051941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.052355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.053173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:01.053610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.055134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.061253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 462/912] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:01.067026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:01.097954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ba35e-1e1a-4609-b2b0-c88ce25d95be None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.142628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.142816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:01.144132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.145040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.145595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.146746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.148863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:01.165194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:01.166288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:19:01.167177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:01.170368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/913] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Jul 19 12:19:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:01.176930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:01.197313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecf40f-7fa6-4a9e-9c0c-9fd91e57184d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.241594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.242808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.243523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.243947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.244855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.247368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.261386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:01.271348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 463/914] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:19:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:19:01.303600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271fd2e-f779-4b00-99c9-721c77999566 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.349408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.350945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.351435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.351945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.352904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:01.353423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.355245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.366360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/915] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:01.368517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:01.492114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b39326-c7f3-4663-b25c-49a4d93623dc None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.508893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:01.533718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.534391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.534677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.534958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.535670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:01.535978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.537451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.544262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 464/916] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:01.548652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:01.665600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da649a-7e73-4955-afa8-77267db0c17b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.694550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:01.704102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.704802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.705091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.705372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.706167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] RBAC: Adding query filter params (interface=public, service_id=b032f3d3d0844b7a9b76a5e3b9bfd1aa, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:01.706483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.707855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207fe8f-9f9b-4d17-8c4b-723d1a0a6789 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.713663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/917] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:19:01 2022] GET /identity/v3/endpoints?service_id=b032f3d3d0844b7a9b76a5e3b9bfd1aa&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:01.718992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:01.833604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3134-0e1d-4eb0-b40c-4bb63bce931a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:01.871521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eef225-29ee-419b-befa-0dcaca5afc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:01.872659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eef225-29ee-419b-befa-0dcaca5afc76 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:01.873169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eef225-29ee-419b-befa-0dcaca5afc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:01.873694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eef225-29ee-419b-befa-0dcaca5afc76 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:01.874593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eef225-29ee-419b-befa-0dcaca5afc76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:01.876232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eef225-29ee-419b-befa-0dcaca5afc76 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:01.882859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 465/918] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:19:01 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:19:02.439511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824eb3d-37d6-44b0-9c06-e0812a309994 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:02.440028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824eb3d-37d6-44b0-9c06-e0812a309994 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:02.440133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824eb3d-37d6-44b0-9c06-e0812a309994 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:02.441673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/919] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:02.447006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:02.447300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:02.447579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:02.464971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:02.466143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:02.506141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:02.551793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 466/920] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:02.574042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824eb3d-37d6-44b0-9c06-e0812a309994 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:02.685367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824eb3d-37d6-44b0-9c06-e0812a309994 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:02.720049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:02.720802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:02.721085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:02.721361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:02.722012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:02.723369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:02.735471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:02.736547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:02.737419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/921] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:02.743437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:02.850515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51688e-972c-4f9d-a899-70e1567075f5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:02.886920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:02.887976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:02.888487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:02.888987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:02.889892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:02.890394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:02.891978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77960cc2-4a12-413c-b9a5-fa533a2ea940 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:02.897715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 467/922] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:02.903182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:02.962629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.010514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b32090-dfb1-4940-9419-1f4a76740d1c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.048918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.049143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902df16-3070-4b64-bec8-e8c3810564f5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.049393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902df16-3070-4b64-bec8-e8c3810564f5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.049649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902df16-3070-4b64-bec8-e8c3810564f5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.050503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.050992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.051255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 468/923] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:03.051669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.052619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:03.053133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.054713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:03.056450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.056759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.057080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.061776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/924] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:03.069155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.077408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:03.078953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:03.112209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.156970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 469/925] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:03.172050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24c4e-b15a-4e91-a4bf-babe0f64cf2c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.181737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.192520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.208252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.208760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.209045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.209323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.210165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] RBAC: Adding query filter params (interface=public, service_id=8df716d82cf940169aecf9b1648a02cc, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:03.210408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.211814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:03.217423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/926] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/endpoints?service_id=8df716d82cf940169aecf9b1648a02cc&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:03.222827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.294152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aaf82c-9ecb-45b3-88c2-f21c0b36810a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.324893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264af6f3-524d-47cd-81b1-0842890b5f15 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.333230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.334442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.334940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.335473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.347060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:03.356367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:03.357281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:03.358107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:03.360303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 470/927] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:03.365074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35140e-f415-48ec-b776-1618c483b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.365074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35140e-f415-48ec-b776-1618c483b602 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.365074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35140e-f415-48ec-b776-1618c483b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.365074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35140e-f415-48ec-b776-1618c483b602 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.365427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35140e-f415-48ec-b776-1618c483b602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.365427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35140e-f415-48ec-b776-1618c483b602 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:03.366688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.376338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/928] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:19:03.400728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.486623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe0dcc-618c-471b-bbb6-05598206afe0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.487170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe0dcc-618c-471b-bbb6-05598206afe0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.487170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe0dcc-618c-471b-bbb6-05598206afe0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.488955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/929] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:19:03.489260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc14e6-c448-4b63-9c07-5031f79c7b28 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.494880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.495702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.495975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.513364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:03.514596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:03.533727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.534681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.534971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.535246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.536003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:03.536341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.538148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:03.548724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 471/930] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:03.554303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.576350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.616227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/931] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:03.648615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.673490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.691962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f086ce-547e-48db-b340-aac44bc60ba7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.743748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.745314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.745753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.746198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.760339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1144dc0-2b9a-4243-86f5-7553efc1c624 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.764334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.765827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:03.792111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:03.793081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:03.793939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:03.793939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:03.793939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:03.793939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:03.796376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 472/932] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:03.801911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.805681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.806425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.806787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.807158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.807982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.809573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:03.811027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING keystone.server.flask.common [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 19 12:19:03.831771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 12:19:03.831771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 12:19:03.831771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [parameters: {'id': '2c7163a49cca493f8869052ee999cbf5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 12:19:03.831771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2c7163a49cca493f8869052ee999cbf5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2c7163a49cca493f8869052ee999cbf5. Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 12:19:03.843489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 19 12:19:03.844310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 19 12:19:03.845249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 12:19:03.846145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 12:19:03.847104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 12:19:03.847897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': '2c7163a49cca493f8869052ee999cbf5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:03.848782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:03.850919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2c7163a49cca493f8869052ee999cbf5. Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/933] 10.222.0.13 () {60 vars in 1140 bytes} [Tue Jul 19 12:19:03 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:03.853169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:03.897646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:03.946755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe894c-c55c-4319-819f-d9a0bad37fc5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.981664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d65351-0f9d-4881-a61b-b2482aa64d50 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:03.990722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:03.991162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:03.991452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:03.991732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:03.992488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:03.992795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:03.994427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fda7c1-ea87-406e-8ccb-58689b96b9ee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.003626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.008026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 473/934] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:04.009331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2731dc-e41e-4f0f-928c-b20de3ba0ae6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.009970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2731dc-e41e-4f0f-928c-b20de3ba0ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.010254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2731dc-e41e-4f0f-928c-b20de3ba0ae6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.011632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 474/935] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:04.013108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2731dc-e41e-4f0f-928c-b20de3ba0ae6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.022467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.023979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.024502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.025007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.032898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.034753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e50c66fa-afee-4e45-b712-ec06be9959f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:04.049919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:04.050875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:04.051763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/936] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:04.052927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.052927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.053107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.069610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:04.070973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:04.129749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2731dc-e41e-4f0f-928c-b20de3ba0ae6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.140797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/937] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:04.141457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.169093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.179602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.180442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.180726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.181005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.188809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.190251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:19:04.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:04.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:04.207025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 475/938] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:04.210620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.252610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a90cb-c008-488b-9c97-61cac5ddeeac None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.272833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.288651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.289313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.289600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.289876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.290624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:04.290932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.292441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.312579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/939] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:04.313316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.327730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d756371-31af-49a8-8d38-0be712c4330e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.368477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.368762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.369040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.369816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.371910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:04.385828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:04.387548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 19 12:19:04.388647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:04.392847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 476/940] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:04.394247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.420881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.441437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4f917-d4b1-4a05-b99c-cc329ce07f0f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.480950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.482026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.482459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.482892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.484012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:04.484537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.486014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.515679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/941] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:04.517639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.529695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cabd51-98f7-4017-ba86-73dc42e956e4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.558571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.570217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.570823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.571108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.571387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.572104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:04.572448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.574361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.579539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 477/942] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:04.585310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.667566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bad75-fe2b-4c81-94a4-565190c09472 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.703839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7163668-d0d8-4d04-97fe-a2764d28f22c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.707962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.708884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.709288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.709664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.720439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.721993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.724024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:04.742267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:04.743694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:04.744674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:04.744674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:04.744674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:04.744674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:04.745577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/943] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:04.746688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.747513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.747924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.748375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.749212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:04.749719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.750835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.751436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.757480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 478/944] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:04.763436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:04.823420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:04.883275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36bac0-153e-406a-bee8-1837d7ec74f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.886657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a1609-3b39-4f88-bdd5-7daeeeb630eb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:04.924527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.925279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.925629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.925996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.926974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.928621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.933633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:04.934786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:04.935257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:04.935716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:04.936895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:04.937212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:04.938562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:04.952217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/945] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:04.957081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 479/946] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:19:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:04.959139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:05.021941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:05.073673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700f9f-cdc8-436f-be5b-a292839e236b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:05.116018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:05.116899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.117186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.117462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.125922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:05.127642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:05.144278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:05.145229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:05.146117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/947] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:05.153324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:05.268757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5281d4-6174-4e02-b766-24cf8fd999b4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:05.278373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:05.306939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:05.307972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.308428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.308921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.309854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:05.310304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:05.311923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47da58-0301-448c-b9c2-a1cde81a4041 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:05.325929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 480/948] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:19:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:05.336200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:05.439838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c4b9b-98c1-4138-9690-5fd1fef1cae8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:05.450973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:05.468630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd1f7-263b-4461-938d-5ca275bbbc06 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.468945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd1f7-263b-4461-938d-5ca275bbbc06 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.469188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcd1f7-263b-4461-938d-5ca275bbbc06 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.470757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 481/949] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:05.475731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.476019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.476281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:05.476417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.477167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.477449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.477727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/71391e0896644c0a82a06b0905b9df24/roles/f79951bd70da4fb6b183f9f4be427b56` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.480589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=71391e0896644c0a82a06b0905b9df24, role_id=f79951bd70da4fb6b183f9f4be427b56)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:05.482025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:05.494238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:05.495489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:05.499617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/950] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:19:05 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/71391e0896644c0a82a06b0905b9df24/roles/f79951bd70da4fb6b183f9f4be427b56 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:19:05.582747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 482/951] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:05.617185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:05.730440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8f5ef-0839-40dd-be2c-8052827a51ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:05.769835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:05.770995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.771465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.771952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.772825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:05.774381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:05.787744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:05.788788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:05.789636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:05.792128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/952] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:05.800691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:05.909746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:05.913826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39805c75-500f-4d1d-9819-fd7cd83318a2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:05.949823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:05.950497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:05.950786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:05.951065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:05.951774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:05.952079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:05.953482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:05.958566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 483/953] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:05.964257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:06.002751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:06.080931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692dc64-0eff-4453-9912-382d11da9ea5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:06.115674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:06.116377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:06.116665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:06.116944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:06.117658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:06.118288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:06.119719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a67e64-5fb8-4c44-bf20-8ab788b50f1e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:06.125231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/954] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:06.130861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:06.243751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058c34f-8c86-429e-9b58-ca36076227ac None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:06.281091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1b290-3bf5-445c-83e2-125707339041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:06.281764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1b290-3bf5-445c-83e2-125707339041 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:06.282049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1b290-3bf5-445c-83e2-125707339041 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:06.282328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1b290-3bf5-445c-83e2-125707339041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:06.283085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1b290-3bf5-445c-83e2-125707339041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:06.284497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1b290-3bf5-445c-83e2-125707339041 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:06.307699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 484/955] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:19:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:06.646793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b4ae-abac-4c50-b8fb-e7d480d668a9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:06.647346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b4ae-abac-4c50-b8fb-e7d480d668a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:06.647346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52b4ae-abac-4c50-b8fb-e7d480d668a9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:06.649128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/956] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:06.654357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:06.654781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:06.654920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:06.672554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:06.673744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:06.689012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:06.756876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 485/957] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:06.782118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b4ae-abac-4c50-b8fb-e7d480d668a9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:06.890923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52b4ae-abac-4c50-b8fb-e7d480d668a9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:06.892461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:06.928266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:06.928937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:06.929226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:06.929504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:06.930263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:06.931694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:06.945750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 12:19:06.945750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 12:19:06.945750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [parameters: {'user_id': '61d9c86d211e4c5ca4de9e41ddd1793c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 12:19:06.945750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0dc29602-a455-45e7-8e9d-1d74a993f5b3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 19 12:19:06.954133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 12:19:06.954992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 19 12:19:06.955956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 19 12:19:06.957431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 19 12:19:06.958396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 19 12:19:06.959245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'user_id': '61d9c86d211e4c5ca4de9e41ddd1793c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:06.960129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 19 12:19:06.962474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:06.964767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/958] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Jul 19 12:19:06 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:06.972339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:07.070976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d809a0-acfd-457b-b6ed-0ad2b0ba3553 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:07.106958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:07.107766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.108054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.108366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.115767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:07.117261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:07.130698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60ea32-f1e0-4274-a964-62d05ae83b01 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.130982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60ea32-f1e0-4274-a964-62d05ae83b01 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.131257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60ea32-f1e0-4274-a964-62d05ae83b01 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:19:07.132970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:07.135079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:07.137975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 486/959] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:07.144294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/960] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:19:07.145133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:07.146214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.146759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.147240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.149294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:07.167938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:07.169583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:07.244224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/961] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:07.260016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5feab-7ec0-4f61-9e01-b3a3651d1a7d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:07.266271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:07.301808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:07.303656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.304224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.304708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.305806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:07.306316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:07.308668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:07.364824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 487/962] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Jul 19 12:19:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:07.370891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:07.381050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9356a2-02ad-4dcd-b2b0-7fe7b821e2c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:07.420596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:07.421311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.421594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.421869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.422530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:07.424045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:07.437073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:07.438170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:07.439016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:07.443348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/963] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:07.444884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:07.516043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:07.558390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f9af4-c6a7-4ff1-b73c-63dac2dd6e9e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:07.594018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:07.594688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.595024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.595346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.596053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:07.596385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:07.597751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fd25a-f676-43b1-ae19-225f8d25cdc2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:07.603073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 488/964] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:07.615192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:07.725541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcba4a-1c10-44ed-a2b2-5f6523f3e560 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:07.758589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:07.759243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.759533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.759809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.760577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:07.760945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:07.762309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe796025-ab4a-4768-9fa0-5b8c121c2973 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:07.767275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/965] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:07.811894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dfea3-98d1-48f6-9f2d-30e66de9ee31 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.812440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dfea3-98d1-48f6-9f2d-30e66de9ee31 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.812541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dfea3-98d1-48f6-9f2d-30e66de9ee31 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.814133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 489/966] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:07.819905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:07.820237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:07.820531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:07.838436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:07.840258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:07.855326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:07.915962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/967] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:07.942346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dfea3-98d1-48f6-9f2d-30e66de9ee31 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.054644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dfea3-98d1-48f6-9f2d-30e66de9ee31 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.079953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:08.090111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.090929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.091214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.091492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.101942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:08.110137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:08.111015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:08.111872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:08.116232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 490/968] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jul 19 12:19:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:08.117483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.217519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f029c2-f8e4-4245-8e24-e9c9910924f3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.253720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.254390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.254674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.254946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.255688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:08.255992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.257376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.271041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/969] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:08.272279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.372156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fcce5-fe90-474d-a31c-63bd8aebeb0c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.405377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.406166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.406447 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.406719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.417792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.419241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d202d964-fe46-410a-a75f-c09cf3b9427a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:08.436303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:08.437169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:08.438040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:08.438040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:08.438040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:08.438040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:08.438040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 491/970] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:08.444252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.459129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:08.463112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd45e-d675-4970-b2af-43862709cfbf None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.464380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd45e-d675-4970-b2af-43862709cfbf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.464663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edd45e-d675-4970-b2af-43862709cfbf None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.466218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 492/971] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:19:08.471045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.471331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.471608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.488313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:08.489502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:08.519798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:08.557976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 493/972] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:08.564833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b5c6b-c1b1-475d-90c3-f9a278af1bf2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.580458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.603095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.603752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.604035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.607275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.607275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:08.607275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.607275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.628313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/973] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:08.629367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.688896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6247a0-e334-46df-82b8-5099303145f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.703428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:08.723998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.724839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.725125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.725409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.726060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.727436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.738379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6f0ae-0e38-4256-926d-322359bf2c96 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:08.740074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:08.741187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:08.742021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:08.747022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 494/974] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:08.747713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.779456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:08.780857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.781694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.781946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.782219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.790672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.792030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:19:08.806997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:08.808760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:08.809702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/975] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:08.814445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.860855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23e754-57f7-4d5e-a055-0ac2e619d3ed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.900841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.901981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.902267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.902540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.903246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:08.903552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.904939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.910146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 495/976] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:08.915593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:08.921624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903e255-c226-4f35-8da9-a9a89ea86434 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:08.954095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:08.960712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:08.961660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:08.961984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:08.962649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:08.963398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:08.963850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:08.966294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:08.984850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/977] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:08.990709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.018139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:09.024344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94757ce-d7e0-4784-a34f-49f05d45c123 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.060593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.061237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.061519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.061795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.062493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:09.062800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.064163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.068964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 496/978] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:09.075254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.097464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf79fa-4e74-4da0-ad29-fcd4cc53fb0e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.131608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.132366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.132656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.132931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.179253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0e82b-f93a-453c-a11e-aca9077d8b23 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.190297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:09.204553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.206034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.216838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.217809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.218100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.218385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.219194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] RBAC: Adding query filter params (interface=admin, service_id=8df716d82cf940169aecf9b1648a02cc, region_id=RegionOne) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:09.219513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.220918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.226079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 497/979] 10.222.0.13 () {56 vars in 1249 bytes} [Tue Jul 19 12:19:09 2022] GET /identity/v3/endpoints?service_id=8df716d82cf940169aecf9b1648a02cc&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:09.231997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.259766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:09.325075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/980] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:08 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:09.327087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.337717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad97742-3025-4cbf-876f-b6eed406ec55 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.371986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.372678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.372959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.373231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.373908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.375278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.381398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 498/981] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:19:09 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:19:09.420154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:09.436078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947cff5-79f7-4f97-80f2-8b299f77fa4f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.473575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.474839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.475302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.475764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.488893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.490504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:09.511933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:09.512963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:09.513870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:09.513870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:09.513870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:09.513870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:09.513870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/982] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:09.520331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.620627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:09.634073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c6e37-f1c3-4393-943f-2c4f568b5408 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.670357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.671252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.671714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.672144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.673041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:09.673501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.675072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f95f-a7f2-4fad-9753-6df283f1994c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.690239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 499/983] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:09.695940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.799893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3872c-a22b-4f98-bc52-51d4f85a6ea7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:09.835477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:09.836253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c035a1b-8401-4523-9c0c-15725c3cb820 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.836367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.836548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c035a1b-8401-4523-9c0c-15725c3cb820 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.836646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.836835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c035a1b-8401-4523-9c0c-15725c3cb820 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.836932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.838369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 500/984] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:09.844066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:09.844753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:09.845267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:09.846234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:09.848715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:09.851949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:09.865073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:09.866273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:09.869805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:09.871373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:09.872541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:09.876264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/985] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:09.877167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.941447 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 501/986] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:09.963365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:09.981716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12d3eb-afde-4a96-8314-620280a0b1a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:10.019995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:10.020958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:10.021252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:10.021532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:10.022319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:10.022631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:10.024069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:10.038319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/987] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:19:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:10.045066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:10.047001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:10.070842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1bc74-dc5e-4dd8-829c-0872485b1b78 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:10.107488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41bd8f-64a3-444d-8598-cdfb6a4f745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:10.107854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:10.108162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41bd8f-64a3-444d-8598-cdfb6a4f745f None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:10.108478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41bd8f-64a3-444d-8598-cdfb6a4f745f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:10.108759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41bd8f-64a3-444d-8598-cdfb6a4f745f None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:10.109498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41bd8f-64a3-444d-8598-cdfb6a4f745f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:10.110977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41bd8f-64a3-444d-8598-cdfb6a4f745f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:10.125505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 502/988] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:19:09 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:10.156940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71158c-21b1-4eee-a82b-fa6264cfde91 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:10.202816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:10.203398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:10.203707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:10.203985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:10.204855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] RBAC: Adding query filter params (role.id=f79951bd70da4fb6b183f9f4be427b56, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=71391e0896644c0a82a06b0905b9df24) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:10.205168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:10.206602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30a14d-3262-4dce-b99a-5838f82a3237 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:10.220220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/989] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:10 2022] GET /identity/v3/role_assignments?user.id=71391e0896644c0a82a06b0905b9df24&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f79951bd70da4fb6b183f9f4be427b56 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:11.406983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50818-2d51-46e4-8243-462f48a9147d None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:11.407533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50818-2d51-46e4-8243-462f48a9147d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:11.407533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50818-2d51-46e4-8243-462f48a9147d None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:11.409088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 503/990] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:11.414008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:11.414383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:11.414569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:11.431536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:11.432797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:11.498939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/991] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:11.521440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50818-2d51-46e4-8243-462f48a9147d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:11.632652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50818-2d51-46e4-8243-462f48a9147d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:11.667677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:11.668450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:11.668740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:11.669021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:11.669681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:11.671068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:11.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:11.684202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 19 12:19:11.685051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:11.688322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 504/992] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Jul 19 12:19:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:11.690792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:11.748328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:11.800950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc6600-b01a-428a-944d-06da321dd81b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:11.835570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:11.836270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:11.836556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:11.836831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:11.837534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:11.837840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:11.839703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:11.849384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/993] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:11.850495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:11.853376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:11.955178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca34fe-58d5-419f-9a50-f8db85aef802 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:11.988527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:11.989190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:11.989471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:11.989803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:11.990534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:11.990841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:11.992219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f77de-3733-4341-8690-5460d5b4c03b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:11.998040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 505/994] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Jul 19 12:19:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:12.003609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.102350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0054ad1-8b72-4c35-9f90-02a2d8632b50 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.116623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.137655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3e302-ea68-4e1f-b215-d2304c81e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.138310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e302-ea68-4e1f-b215-d2304c81e0d8 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.138592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e302-ea68-4e1f-b215-d2304c81e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.138864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e302-ea68-4e1f-b215-d2304c81e0d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.139609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3e302-ea68-4e1f-b215-d2304c81e0d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.140994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3e302-ea68-4e1f-b215-d2304c81e0d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.157295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/995] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:19:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:12.210176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330e7b2-3e8c-400f-a587-df8c62cab1d6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.210369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330e7b2-3e8c-400f-a587-df8c62cab1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.210658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330e7b2-3e8c-400f-a587-df8c62cab1d6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.212296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 506/996] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:12.217848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.218149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.218434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.236627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:12.237859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:12.283258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774fb2-4f26-433d-a90f-4e0a39b21190 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.283613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774fb2-4f26-433d-a90f-4e0a39b21190 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.283811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774fb2-4f26-433d-a90f-4e0a39b21190 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.285390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 507/997] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:12.290769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.291058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.291337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.310419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:12.311014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:12.319862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/998] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:12.343551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.382277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.390909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 508/999] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:12.413620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.454754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.459197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e88bb2-2aa6-41b7-aa21-d2f1a05493a3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.497940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.498838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.499130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.499397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.500044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.501550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.513986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252d776-4fbe-46b7-a5eb-0a4ac4e1d74a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:12.514640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:12.515612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:12.516487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/1000] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:12.521986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.552655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.553453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.553750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.554019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.554686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.556055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:12.568508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:12.569740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:12.570876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:12.574648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 509/1001] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:12.575485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.635651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a17c-69a1-446f-b830-8f627d8878b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.644400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.673381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.674052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.674344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.674627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.675339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:12.675652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.677058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.682051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/1002] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:12.687545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.697488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f2c8-57b2-4642-9952-5855afbde199 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.717716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.736370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.737044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.737411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.737614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.738328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] RBAC: Adding query filter params (name=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:12.738931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.740018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.745563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 510/1003] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:12.751185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.797000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09fbc9-ba53-4d26-b49e-3a44a57098d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.831271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.831940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.832256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.832541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.833249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:12.833556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.834924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.844599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/1004] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:12.845551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:12.864875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34a9fb-c273-490e-9f34-26440957c741 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.889842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.902639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.903361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.903752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.903929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.904667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:12.904980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.906358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:12.911305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 511/1005] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:12.947448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:12.957818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee29a2-1c00-44ae-a586-d55b52c7a702 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:12.995861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:12.996558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:12.996841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:12.997118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:12.997912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] RBAC: Adding query filter params (interface=public, service_id=767bc88e64c44383b5912c2e64b9aeb5, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:12.998224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:12.999604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392855c-d81d-428f-971f-9c8a45abded9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:13.005884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/1006] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:19:12 2022] GET /identity/v3/endpoints?service_id=767bc88e64c44383b5912c2e64b9aeb5&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:13.011267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:13.119845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474661d4-d0bb-4549-8ba7-1f63ac1ceaa2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:13.154359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81cceb-be32-4aed-b26e-89737787e34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:13.155033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81cceb-be32-4aed-b26e-89737787e34e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:13.155320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81cceb-be32-4aed-b26e-89737787e34e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:13.155600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81cceb-be32-4aed-b26e-89737787e34e None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:13.156322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81cceb-be32-4aed-b26e-89737787e34e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:13.157704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81cceb-be32-4aed-b26e-89737787e34e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:13.169383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 512/1007] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:19:13 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:19:14.214079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7311-3649-4789-a055-630bf8925dbc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:14.214594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7311-3649-4789-a055-630bf8925dbc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:14.214671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d7311-3649-4789-a055-630bf8925dbc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:14.216415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/1008] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:14.223564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:14.224308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:14.224308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:14.243995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:14.246602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:14.305505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:14.336527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 513/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:14 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:14.354895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d7311-3649-4789-a055-630bf8925dbc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:14.457143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d7311-3649-4789-a055-630bf8925dbc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:14.490365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97870b36-04ed-4579-97d5-3c88a14949d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:14.491153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97870b36-04ed-4579-97d5-3c88a14949d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:14.491440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97870b36-04ed-4579-97d5-3c88a14949d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:14.491720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97870b36-04ed-4579-97d5-3c88a14949d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:14.493170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97870b36-04ed-4579-97d5-3c88a14949d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:14.494655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97870b36-04ed-4579-97d5-3c88a14949d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:14.501367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/1010] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:14.502881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:14.606403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cdff9-e420-45ed-a433-239eac86bee3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:14.652749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951ff5c-b4ef-47aa-87fe-60a75e028a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:14.653422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951ff5c-b4ef-47aa-87fe-60a75e028a79 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:14.653709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951ff5c-b4ef-47aa-87fe-60a75e028a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:14.653989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951ff5c-b4ef-47aa-87fe-60a75e028a79 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:14.654714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951ff5c-b4ef-47aa-87fe-60a75e028a79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:14.656152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951ff5c-b4ef-47aa-87fe-60a75e028a79 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:14.692218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 514/1011] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:19:14 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:14.938820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e237-2c5b-4c5c-8a30-564ec24329ab None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:14.939010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e237-2c5b-4c5c-8a30-564ec24329ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:14.939297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e237-2c5b-4c5c-8a30-564ec24329ab None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:14.940866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/1012] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:14.945824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca6ec-a9e1-45cb-8de2-49d8245241c3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:14.946190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca6ec-a9e1-45cb-8de2-49d8245241c3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:14.946564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca6ec-a9e1-45cb-8de2-49d8245241c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:14.963758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-83aca6ec-a9e1-45cb-8de2-49d8245241c3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:14.965064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-83aca6ec-a9e1-45cb-8de2-49d8245241c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:14.993768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:15.037689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 515/1013] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:14 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:15.061105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644e237-2c5b-4c5c-8a30-564ec24329ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:15.088603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:15.167485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644e237-2c5b-4c5c-8a30-564ec24329ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:15.202591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021cf14-9788-4047-80f8-580e2378c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:15.203603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021cf14-9788-4047-80f8-580e2378c48f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.204097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021cf14-9788-4047-80f8-580e2378c48f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.204647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021cf14-9788-4047-80f8-580e2378c48f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.205580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021cf14-9788-4047-80f8-580e2378c48f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:15.207140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021cf14-9788-4047-80f8-580e2378c48f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:15.222040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/1014] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:19:15 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:15.230188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b5ca5-ed79-4398-a583-af3b0167f7a6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.230745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b5ca5-ed79-4398-a583-af3b0167f7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.230745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b5ca5-ed79-4398-a583-af3b0167f7a6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.232317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 516/1015] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:15.238147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.238458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.238727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.257763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:15.259016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:15.348158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/1016] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:15 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:15.367343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b5ca5-ed79-4398-a583-af3b0167f7a6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:15.473663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b5ca5-ed79-4398-a583-af3b0167f7a6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:15.508063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:15.508895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.509183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.509463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] PATH_INFO: `/v3/services/compute` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.510144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:15.511526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:15.523193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:15.524211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:15.525087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 517/1017] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:15.533014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:15.545491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:15.639217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a5192-f97d-4c94-8135-b7f87deb2b4c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:15.677078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:15.677754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.678042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.678322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.679038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:15.679349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:15.680766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:15.691158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/1018] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:15.692436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:15.723055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:15.799187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5b37c-0940-4a20-965a-ad878a04bede None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:15.833608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:15.834298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.834584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.834860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.835590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:15.835901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:15.837359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309644e6-e5fb-42e7-a0eb-f5fa2804dde9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:15.847959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 518/1019] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Jul 19 12:19:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:15.849038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:15.930360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:15.959007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51e760-4429-4454-8234-e62bed32d974 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:15.993498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ffaa05-4ec9-4c78-b454-deb4313c537b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:15.994206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffaa05-4ec9-4c78-b454-deb4313c537b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:15.994526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffaa05-4ec9-4c78-b454-deb4313c537b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:15.995341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ffaa05-4ec9-4c78-b454-deb4313c537b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:15.996085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ffaa05-4ec9-4c78-b454-deb4313c537b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:15.997464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ffaa05-4ec9-4c78-b454-deb4313c537b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:16.020216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:19:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:16.756829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc61735-77b2-4293-bec2-a25785c60547 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:16.757335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc61735-77b2-4293-bec2-a25785c60547 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:16.757446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc61735-77b2-4293-bec2-a25785c60547 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:16.758728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 519/1021] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:16.764217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:16.764619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:16.764789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:16.782505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:16.783691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:16.860290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:16.884886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc61735-77b2-4293-bec2-a25785c60547 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:16.981494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:16.992504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc61735-77b2-4293-bec2-a25785c60547 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:17.027094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d49e4-6d7c-4971-ba50-74e49a4d5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:17.027889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d49e4-6d7c-4971-ba50-74e49a4d5ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.028199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d49e4-6d7c-4971-ba50-74e49a4d5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.028481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d49e4-6d7c-4971-ba50-74e49a4d5ba0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.029644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d49e4-6d7c-4971-ba50-74e49a4d5ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:17.031083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d49e4-6d7c-4971-ba50-74e49a4d5ba0 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:17.032930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 520/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:17.038364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:17.094851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:17.141240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7854a7b4-064f-4675-ab3f-4a2e484ec7b4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:17.174714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3c678-4713-486a-8d27-a586bd31698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:17.175381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3c678-4713-486a-8d27-a586bd31698b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.175668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3c678-4713-486a-8d27-a586bd31698b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.175943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3c678-4713-486a-8d27-a586bd31698b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.176689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3c678-4713-486a-8d27-a586bd31698b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:17.178119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3c678-4713-486a-8d27-a586bd31698b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:17.204268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:19:17 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:17.278520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b42344-1f74-45f9-a82c-9803bda83172 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.278841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b42344-1f74-45f9-a82c-9803bda83172 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.279081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b42344-1f74-45f9-a82c-9803bda83172 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.280701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 521/1025] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:17.286258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.286542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.286819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.304925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:17.306148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:17.382393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1026] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:17.405572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b42344-1f74-45f9-a82c-9803bda83172 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:17.510827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b42344-1f74-45f9-a82c-9803bda83172 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:17.543446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:17.544274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.544564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.544844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.545494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:17.546865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:17.558814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:17.559786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:17.560684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 522/1027] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Jul 19 12:19:17 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:17.565839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:17.614833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:17.679712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32976e-8358-4d69-a2aa-53a27161cc7b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:17.715230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:17.715905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.716211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.716499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.717205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:17.717515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:17.718879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:17.724246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1028] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:17.730174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:17.759761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:17.835693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6f94f-4113-4813-9886-b1edac1dd934 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:17.870058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:17.870722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:17.871005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:17.871282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:17.871983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] RBAC: Adding query filter params (type=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:17.872313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:17.873688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:17.883330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 523/1029] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:17.884608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:17.986770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9747ff-61e3-4f32-a041-670386cac778 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:17.994511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:18.023302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:18.023961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.024278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.024561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.025351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] RBAC: Adding query filter params (interface=public, service_id=c8f071fe5e12485ca0a5bf443f215856, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:18.025692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:18.027593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e52cb2-0662-4e93-84b7-49989e4045ff None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:18.037354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1030] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:19:17 2022] GET /identity/v3/endpoints?service_id=c8f071fe5e12485ca0a5bf443f215856&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:18.038880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:18.141170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:18.154789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dc96c-8203-42e9-b470-d33f610f0b5e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:18.187747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07da46-35ca-4715-a748-2a9f86a3b902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:18.188445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07da46-35ca-4715-a748-2a9f86a3b902 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.188734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07da46-35ca-4715-a748-2a9f86a3b902 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.189014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07da46-35ca-4715-a748-2a9f86a3b902 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.189699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07da46-35ca-4715-a748-2a9f86a3b902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:18.191074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07da46-35ca-4715-a748-2a9f86a3b902 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:18.204211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 524/1031] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:19:18 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:19:18.244074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92243b9-c360-4567-9c0b-31ed74b42457 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.244881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92243b9-c360-4567-9c0b-31ed74b42457 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.245137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92243b9-c360-4567-9c0b-31ed74b42457 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.246819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1032] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:18.252401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.252747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.252981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.271228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:18.272496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:18.347064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 525/1033] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:18.370849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92243b9-c360-4567-9c0b-31ed74b42457 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:18.482190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92243b9-c360-4567-9c0b-31ed74b42457 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:18.516473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:18.517237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.517522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.517795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.528553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:18.536797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:18.537726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:18.544165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1034] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jul 19 12:19:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:18.546262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:18.636265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:18.655455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe68b8-5143-48a6-8c24-d58b7674d9f9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:18.691929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:18.692634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.692917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.693193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.693942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:18.694247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:18.695616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:18.707995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 526/1035] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:19:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:18.709778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:18.742028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:18.814946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d188fdb-5552-4862-bfe2-61ed3de4e23c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:18.847881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:18.848706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:18.848996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:18.849280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:18.861211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:18.862660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:18.882176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:18.883080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:18.883964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:18.883964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:18.883964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:18.883964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:18.888484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:18.890084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:18.991945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14829b-a910-44c4-8d2d-93fea2257598 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.025277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.026549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.026835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.027110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.027824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:19.028131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.029535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.046305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 527/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:19.047823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.129793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.149062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda4192-d11b-4bc2-801a-f2a38cc6e7b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.183028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.183809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.184093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.184393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.192025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.193481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1d96636b-0169-42fd-ba9a-6e2544fafeaa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:19:19.207805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:19.208801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:19.213814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:19.217337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.221171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.325983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c4c12-afe6-4e4d-96d4-70a43bf5de16 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.341312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d47f-77d4-44a3-96d1-93d55774c0b3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.341645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d47f-77d4-44a3-96d1-93d55774c0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.341880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d47f-77d4-44a3-96d1-93d55774c0b3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.343501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1039] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:19:19.348794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.349089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.349375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.364819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.365493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.365778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.366053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.366888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:19.367199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.367337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:19.368895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:19.369023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.391703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 528/1040] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:19.397253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.444999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1041] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:19.463031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.467711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.505989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5284d3-6348-4de0-9fd1-e61bc84ad711 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.542800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.545880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.546539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.546828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.547109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.573849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfc435-6ad8-4608-a2ce-2761cde41ae8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.614513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.615317 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.615663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.615954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.624880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.626675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:19.626775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:19.634356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:19.635812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:19.636688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1042] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:19.641969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.693343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.737516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 529/1043] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 12:19:19.743348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.749594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ed80a-a6fa-4326-8343-b52290b714e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.781882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.787514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.788051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.788371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.788710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.789496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:19.789812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.791239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.805597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1044] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:19.806622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.852413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee810f64-0eff-4faf-9035-22e2d3655eee None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.889176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.889965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.890246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.890781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.901777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.903148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.918708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e88986-63e2-4ad0-a3b3-fbc54465bbb7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:19.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:19.921430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:19.922494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:19.922494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:19.922494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:19.922494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:19.922494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 530/1045] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:19.927213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:19.944301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:19.959320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:19.960207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:19.960493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:19.960754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] PATH_INFO: `/v3/users/a1a10d084ab84b4783f4d56ad7ae56c0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:19.975472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a10d084ab84b4783f4d56ad7ae56c0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:19.976936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:19.978602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1046] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/users/a1a10d084ab84b4783f4d56ad7ae56c0 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:19.984148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.036482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29eff44-d7a2-477d-a124-141673e73ed6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.041864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.073591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.074259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.074545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.074823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.075549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:20.075858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.077272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.094326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 531/1047] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:20.095663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.098100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd8a59-8c20-4f2b-ae14-c2ffa5372419 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.138730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec28df-bf73-470b-904b-f076f310cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.139379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec28df-bf73-470b-904b-f076f310cc35 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.139661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec28df-bf73-470b-904b-f076f310cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.139939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec28df-bf73-470b-904b-f076f310cc35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.175865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.201943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3886d-f8b0-43b8-9868-269597fd3d21 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.214670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec28df-bf73-470b-904b-f076f310cc35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.216165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec28df-bf73-470b-904b-f076f310cc35 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.238282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.239028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.239370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.239601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.248397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.249957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-4fa9e04d-a2f0-40cc-9313-ec4a25d82900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:20.264717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:20.265739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:20.266642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 532/1048] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:20.267573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c665f-7829-47ff-bd87-7622c1fe05f1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.267573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c665f-7829-47ff-bd87-7622c1fe05f1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.267867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c665f-7829-47ff-bd87-7622c1fe05f1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.268854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.270736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 533/1049] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:20.271672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c665f-7829-47ff-bd87-7622c1fe05f1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.322256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1050] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:19 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 19 12:19:20.322256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.322256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.322256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.335724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:20.336939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:20.372790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c665f-7829-47ff-bd87-7622c1fe05f1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.402614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1051] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:20.403437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.406307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.406967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.407262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.407542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.408372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:20.408694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.410138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.411951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.423512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 534/1052] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:20.426487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.491296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.510397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c2334-a91a-47c7-b746-33cc98637b6a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.543391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07c4b7-4539-47d7-b5a4-2891b2b4d643 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.544055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.545038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.545387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.545671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.546860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] RBAC: Authorizing `identity:get_project(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.548312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.550029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1053] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:20.550787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.581525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.582309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.582594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.582866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.583512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.584920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:20.596719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:20.597671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:20.598504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 535/1054] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:20.599301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.654328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d6d1e-7b00-4937-adc9-c6b66e3ba149 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.658285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.687878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.688482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.688766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.689042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.689864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] RBAC: Adding query filter params (role.id=f79951bd70da4fb6b183f9f4be427b56, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=99d07f11438d4e069eab093a8b99eeab) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:20.690174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.691599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.697095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1055] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/role_assignments?user.id=99d07f11438d4e069eab093a8b99eeab&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f79951bd70da4fb6b183f9f4be427b56 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:20.697914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.714731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:20.717690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192c20e-fe50-44cb-b55f-5739185b3db8 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.759802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.761649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.761943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.762221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.763133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6, user.id=a1a10d084ab84b4783f4d56ad7ae56c0) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:20.763566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.765350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.784216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 536/1056] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/role_assignments?user.id=a1a10d084ab84b4783f4d56ad7ae56c0&scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:20.827234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13819e-4620-47a4-a2cc-c2c3660d7cd7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:20.866592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:20.867280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:20.867569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:20.868111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:20.868877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:20.869191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:20.871126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:20.881300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:20.883034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:20.990711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026f344-990e-4455-b701-4cecf7d3cfd4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:21.013950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:21.026343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:21.027008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:21.027288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:21.027562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:21.028288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] RBAC: Adding query filter params (type=image) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:21.028596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:21.029968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acd43b-a4ab-4cec-b0f5-b50e35b37276 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:21.039535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 537/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:21.040740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:21.122930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:21.146058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feef19a-8695-4a61-8218-f54b029cd6b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:21.180330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e36370-1721-43fe-9db3-745dfdfbf530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:21.180991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e36370-1721-43fe-9db3-745dfdfbf530 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:21.181280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e36370-1721-43fe-9db3-745dfdfbf530 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:21.181561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e36370-1721-43fe-9db3-745dfdfbf530 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:21.182888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e36370-1721-43fe-9db3-745dfdfbf530 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:21.184331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e36370-1721-43fe-9db3-745dfdfbf530 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:21.204036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:19:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:22.727942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2881bd-5ad2-46da-acc1-c8cd19e9b72b None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:22.728514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2881bd-5ad2-46da-acc1-c8cd19e9b72b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:22.728595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2881bd-5ad2-46da-acc1-c8cd19e9b72b None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:22.730062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 538/1060] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:22.734997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:22.735379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:22.735560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:22.752882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:22.754061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:22.793078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd5a94-7206-4873-bd4b-372d2624f6e9 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:22.793387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd5a94-7206-4873-bd4b-372d2624f6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:22.793639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd5a94-7206-4873-bd4b-372d2624f6e9 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:22.795117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 539/1061] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:22.799711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:22.800004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:22.800319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:22.818477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:22.819668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:22.827328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1062] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:22.849547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:22.891545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 540/1063] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:22.911489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:22.914524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:22.958484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19ff56-6ee7-4243-9982-4bb102c738ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:22.996011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:22.996824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:22.997110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:22.997388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.009267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:23.013149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a51763-b34b-4dfe-8d24-b9a2f5fdb7e5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:23.018005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:23.018912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:23.019734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1064] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jul 19 12:19:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:23.025001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.032486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.053675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.054561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.054755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.055034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.066597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:23.074344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:23.075252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:23.076088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 541/1065] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:19:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:23.081381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.135665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ea282b-d3f6-4114-a53e-36f766f02e74 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.156487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.173099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.173776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.174064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.174349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.175101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:23.175415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.176838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.185470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1066] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:23.191104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.194169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204d479-f158-4c7d-87d8-9eb6966cbbd9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.230291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.230966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.231248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.231524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.232310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:23.232678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.234527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.248489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 542/1067] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:23.249461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.292637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.297826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6674-5c29-44d6-a6f8-ae5215cbc360 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.333246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.334447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.334863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.335094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.347490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.348958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.353867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f648001-72a1-4add-aefc-453b18fada83 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7c47310a-2970-4b3b-bb31-2cae4b96163c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:23.366205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:23.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:23.368625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:23.368625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:23.368625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:23.368625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:23.368625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:23.368948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec1094-3cf4-4e29-9294-8667100ce933 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.369189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec1094-3cf4-4e29-9294-8667100ce933 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.369464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec1094-3cf4-4e29-9294-8667100ce933 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.370722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1069] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:23.373461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec1094-3cf4-4e29-9294-8667100ce933 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.394175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61441143-dadd-4f96-97af-99970218b725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.395053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441143-dadd-4f96-97af-99970218b725 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.395339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441143-dadd-4f96-97af-99970218b725 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.395616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61441143-dadd-4f96-97af-99970218b725 None admin] PATH_INFO: `/v3/users/a1a10d084ab84b4783f4d56ad7ae56c0` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.396830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61441143-dadd-4f96-97af-99970218b725 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a10d084ab84b4783f4d56ad7ae56c0)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.398277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61441143-dadd-4f96-97af-99970218b725 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.399904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 543/1070] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/users/a1a10d084ab84b4783f4d56ad7ae56c0 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:23.401654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.401941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.402216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.411554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.420510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:23.421693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:23.477619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec1094-3cf4-4e29-9294-8667100ce933 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.487771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 544/1071] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:23.488671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.515401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.516064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.516387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.516667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.517387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:23.517694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.519059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.534200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1072] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:23.535008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.542643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.606624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a88b7-54d4-4795-aa6e-7b68d1fe6ff7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.629441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.637394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b104984-d029-4f1f-ab96-df0cd2a7e643 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.641909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.642771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.643051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.643323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.644527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] RBAC: Authorizing `identity:get_project(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.645963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.647790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 545/1073] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:23.648499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.672146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.672760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.673048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.673322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.739036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.740519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.745490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9072b6-53e0-4475-9a87-dd409fb8f84b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.747752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.780007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.780818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.781102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.781440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.788433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.790046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:19:23.804395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:23.806505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 546/1074] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:23.810066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.835452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:23.843661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1075] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:23.844554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.916744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4e8a1-c561-42fe-97c0-9bb86b85d750 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.940663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4450598-741a-4e8b-9a3c-02a623bd46da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:23.951009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.951893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.952191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.952487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6/users/a1a10d084ab84b4783f4d56ad7ae56c0/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.954460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6, user_id=a1a10d084ab84b4783f4d56ad7ae56c0, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.955905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:23.969996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 547/1076] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:19:23 2022] PUT /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6/users/a1a10d084ab84b4783f4d56ad7ae56c0/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:19:23.976718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:23.980667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:23.981427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:23.981635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:23.982051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:23.982772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:23.983139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:23.984697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.004328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:24.004859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1077] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:24.010689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:24.069756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:24.099561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc5a9b-678d-48da-a17d-4dbdbc003ab2 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.122537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047d0d4-287e-4dd6-8b4b-8c73ed483d08 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.150222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.151142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.151431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.151712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.163851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.165385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.166971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.168841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.169353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.169900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.182396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.184146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:24.186010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:24.186893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:24.187784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:24.187784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:24.187784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:24.187784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:24.192907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 548/1078] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:24.193975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:24.214289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:24.215304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:24.216263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:24.216263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:24.216263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:24.216263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:24.217653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1079] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:24.222958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:24.295987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:24.327393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c7922-0055-442e-9aef-533ec770a4c6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.349540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0d023-e7fa-4ced-9eb3-7b546e51870e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.352357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:24.366094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.366765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.367049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.367327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.368375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:24.368687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.370595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.392697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 549/1080] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:24.398773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.399639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.399931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.400236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.401524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:24.401838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.403270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.421317 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1081] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:24.425593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:24.550086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0411b-5a5f-4441-9f85-1eb678ba55b1 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.586256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.587204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.587578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.588606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.591444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:24.596686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.598214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:24.614222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:24.615333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:24.616410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:24.617091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 550/1082] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:24.623209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:24.724298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699ce8f-12f0-48f1-8662-a98a49069ab5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.756388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.757043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.757329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.757610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.758397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:24.758711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.760752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521148-85cf-493a-b8ef-e4b8fd467020 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.772442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1083] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:24.778958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:24.875681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591bf15-5f86-4e75-a628-7afcdd5c6c78 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:24.909326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4912f4-9559-4577-ba5f-00180ecf0cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:24.910204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4912f4-9559-4577-ba5f-00180ecf0cd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:24.910486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4912f4-9559-4577-ba5f-00180ecf0cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:24.910762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4912f4-9559-4577-ba5f-00180ecf0cd3 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/99d07f11438d4e069eab093a8b99eeab/roles/f79951bd70da4fb6b183f9f4be427b56` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:24.912933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4912f4-9559-4577-ba5f-00180ecf0cd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=99d07f11438d4e069eab093a8b99eeab, role_id=f79951bd70da4fb6b183f9f4be427b56)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:24.914376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4912f4-9559-4577-ba5f-00180ecf0cd3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:24.927945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 551/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:19:24 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/99d07f11438d4e069eab093a8b99eeab/roles/f79951bd70da4fb6b183f9f4be427b56 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:19:26.060070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453a7e8-8989-47ef-8c7f-f640dda5fc31 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.060645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453a7e8-8989-47ef-8c7f-f640dda5fc31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.060645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d453a7e8-8989-47ef-8c7f-f640dda5fc31 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.062040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1085] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:26.067174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.067564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.067871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.084240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:26.085416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:26.134446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:26.151877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 552/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:26.175112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453a7e8-8989-47ef-8c7f-f640dda5fc31 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:26.200230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:26.275236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d453a7e8-8989-47ef-8c7f-f640dda5fc31 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:26.309034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:26.309817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.310155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.310386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.320411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:26.328675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:26.329800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:26.330959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1087] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:19:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:26.336037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:26.432306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b8af0-00c0-46cb-9938-f20b2a735512 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:26.465267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:26.465936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.466226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.466499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.467241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:26.467545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:26.468938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:26.476978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 553/1088] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:26.483191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:26.565624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:26.581811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbb6d-c2c8-40fb-8d65-6e87ac2fd345 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:26.618020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad445ae6-6349-44e4-a4d3-890f9ecb0159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:26.618887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad445ae6-6349-44e4-a4d3-890f9ecb0159 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.619220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad445ae6-6349-44e4-a4d3-890f9ecb0159 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.619495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad445ae6-6349-44e4-a4d3-890f9ecb0159 None admin] PATH_INFO: `/v3/users/a1a10d084ab84b4783f4d56ad7ae56c0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.620694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad445ae6-6349-44e4-a4d3-890f9ecb0159 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a10d084ab84b4783f4d56ad7ae56c0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:26.622119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad445ae6-6349-44e4-a4d3-890f9ecb0159 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:26.623741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1089] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:26 2022] GET /identity/v3/users/a1a10d084ab84b4783f4d56ad7ae56c0 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:26.629500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:26.678373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:26.729507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38af1b-8097-4790-b973-d4a7a4befb63 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:26.761822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:26.762467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.762755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.763034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.828205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:26.829597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:26.875915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:26.907635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0d57-82eb-4141-a1c1-3464f5f09260 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.907927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0d57-82eb-4141-a1c1-3464f5f09260 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.908234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f0d57-82eb-4141-a1c1-3464f5f09260 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.909604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1090] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:26.914399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:26.914689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:26.914965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:26.933502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 554/1091] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:26.933669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:26.933669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:26.934487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.001112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1092] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:19:27.023980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.032733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0632-4dc9-48b6-b819-f74a9f7be7af None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.067273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.068124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.068431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.068707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.069874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.071301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.073052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 555/1093] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:26 2022] GET /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:27.079147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.096906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:27.104099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:27.123573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbb02-54c7-42ea-88bc-849dc057bb98 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.157647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.158457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.158853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.159023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.169648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:27.177169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66faf28-f85c-418e-b08a-6082dae53eb5 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:27.177866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:27.178873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:27.179718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1094] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:27.185217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.211789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.212470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.213330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.213612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.214450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6, user.id=a1a10d084ab84b4783f4d56ad7ae56c0) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:27.214760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.216214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.227956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 556/1095] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/role_assignments?user.id=a1a10d084ab84b4783f4d56ad7ae56c0&scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:27.293856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d6793-a9b2-4843-936e-4f42b24240ed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.329529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.330126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.330412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.330690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.331817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:27.332133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.333568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.342588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1096] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:27.348522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.430686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:27.450309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45e0a-0d3f-412c-b6f8-756e7520e24e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.489577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.490370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.490657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.490934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.501597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.503065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:27.519661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:27.520751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:27.521788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:27.521788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:27.521788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:27.521788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:27.521788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 557/1097] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:27.527257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.531086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:27.626764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee540b5f-d565-41b5-9094-f4ca254a7bee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.659705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.660383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.660670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.660947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.661662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:27.661976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.663333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.674990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1098] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:27.680578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.777854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e2fd-b9d9-4a64-af0f-d46156b5e5f4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.805525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:27.811309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.812089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.812401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.812678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.819961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.821425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:19:27.835018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:27.836149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:27.842130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 558/1099] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:27.843188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:27.904107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:27.941309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc8898-dadc-4500-8984-e8f1e2b9efc4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:27.974845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:27.975512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:27.975797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:27.976078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:27.976942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:27.977257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:27.978688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:27.996133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Jul 19 12:19:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:28.001583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:28.098592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b69d3-2417-439f-ad4c-0ce673258d97 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:28.136604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:28.137247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:28.137537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:28.137814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:28.193883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:28.203680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:28.205165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:28.299830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 559/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:28.305491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:28.404740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb69f9-c542-4f41-80cd-7d4868fffb8a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:28.426862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:28.438764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:28.439539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:28.439820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:28.440092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:28.450327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:28.451696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:28.469281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:28.470347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:28.475838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:19:28.475838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:28.475838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:19:28.475838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:28.475838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:28.476108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:28.562938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:28.571593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9592-1784-4f59-9244-1b69457f2f9a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:28.608878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:28.609544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:28.609828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:28.610105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:28.610859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:28.611456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:28.612549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:28.624107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 560/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:28.629642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:28.725961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382668-f5a7-422b-b71e-9056b34d87cd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:28.758880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:28.760223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:28.760514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:28.760788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:28.767263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:28.768736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:19:28.782164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:28.783204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:28.788822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:19:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:19:28.789301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:28.887420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513f0ab-77c8-49cc-9b2c-0a558345f88e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:28.919668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:28.920354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:28.920639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:28.920917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:28.921699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:28.922010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:28.923438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0251e6-d5f1-4f8f-9930-62babdb22bb6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:28.935638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 561/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:28.941790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:28.975497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:29.040618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098ed0a-db0c-4f9a-8daa-b2e1c680961b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:29.072983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:29.073628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:29.073917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:29.074199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:29.075021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] RBAC: Adding query filter params (role.id=f79951bd70da4fb6b183f9f4be427b56, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=99d07f11438d4e069eab093a8b99eeab) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:29.075339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:29.076796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c4460a-f0f8-435f-8ee1-071e3ba5a105 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:29.082482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:28 2022] GET /identity/v3/role_assignments?user.id=99d07f11438d4e069eab093a8b99eeab&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f79951bd70da4fb6b183f9f4be427b56 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:29.280291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da448d6-8507-451c-8010-e82be3f87b42 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:29.280804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da448d6-8507-451c-8010-e82be3f87b42 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:29.280897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da448d6-8507-451c-8010-e82be3f87b42 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:29.282195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 562/1107] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:29.287449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:29.287741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:29.288022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:29.305067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:29.306184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:29.374451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:29.398578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da448d6-8507-451c-8010-e82be3f87b42 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:29.414511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:29.506386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da448d6-8507-451c-8010-e82be3f87b42 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:29.542452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602a196-42f9-42c2-8ec7-c56d2eb25be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:29.543248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a196-42f9-42c2-8ec7-c56d2eb25be8 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:29.543532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a196-42f9-42c2-8ec7-c56d2eb25be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:29.543809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602a196-42f9-42c2-8ec7-c56d2eb25be8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:29.545015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602a196-42f9-42c2-8ec7-c56d2eb25be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:29.546458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602a196-42f9-42c2-8ec7-c56d2eb25be8 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:29.548398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 563/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:29.553822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:29.582332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:29.658506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead42fc2-5705-4edd-86c7-89edebaddc49 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:29.692373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be953ff7-43ca-49d7-8f61-a9df6b53e897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:29.693047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be953ff7-43ca-49d7-8f61-a9df6b53e897 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:29.693333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be953ff7-43ca-49d7-8f61-a9df6b53e897 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:29.693612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be953ff7-43ca-49d7-8f61-a9df6b53e897 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:29.694323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be953ff7-43ca-49d7-8f61-a9df6b53e897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:29.695748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be953ff7-43ca-49d7-8f61-a9df6b53e897 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:29.716052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:19:29 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:31.787920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8c238-817e-42da-b616-a8f8f6cb2f9f None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:31.788457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8c238-817e-42da-b616-a8f8f6cb2f9f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:31.788546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8c238-817e-42da-b616-a8f8f6cb2f9f None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:31.789850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 564/1111] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:31.795595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:31.795961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:31.796145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:31.812558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:31.813724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:31.878147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:31.901260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8c238-817e-42da-b616-a8f8f6cb2f9f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:31.979505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:31.999688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8c238-817e-42da-b616-a8f8f6cb2f9f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:32.032514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:32.033360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:32.033642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:32.033914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] PATH_INFO: `/v3/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:32.035361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] RBAC: Authorizing `identity:get_role(role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:32.036758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:32.038109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 565/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:31 2022] GET /identity/v3/roles/7805367da7144187ae1dd2e09705a569 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:32.044490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:32.058828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:32.142103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f0320-bd00-4b05-8cad-162db2ec17ad None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:32.174218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:32.175071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:32.175352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:32.175626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] PATH_INFO: `/v3/users/349357072f3c4f3e8898b8580dc05bf9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:32.187968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] RBAC: Authorizing `identity:get_user(user_id=349357072f3c4f3e8898b8580dc05bf9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:32.189439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:32.191054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:32 2022] GET /identity/v3/users/349357072f3c4f3e8898b8580dc05bf9 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:32.197314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:32.294067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e75d02-7aec-44e7-8486-edb98b79b789 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:32.329894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:32.330544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:32.330828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:32.331105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:32.349023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:32.396658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:32.398057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:32.496817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 566/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:32.502702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:32.570752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:32.599158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175f1d6-5409-4825-9d38-85435b06bc78 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:32.632635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77766e8d-c032-46a3-a5dd-3bbedce42cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:32.633482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77766e8d-c032-46a3-a5dd-3bbedce42cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:32.633765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77766e8d-c032-46a3-a5dd-3bbedce42cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:32.634042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77766e8d-c032-46a3-a5dd-3bbedce42cc6 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:32.635786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77766e8d-c032-46a3-a5dd-3bbedce42cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:32.637232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77766e8d-c032-46a3-a5dd-3bbedce42cc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:32.638895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:32 2022] GET /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:32.644937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:32.691933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:32.740926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f72d9-9e87-43d3-a36e-73d000b87f12 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:32.778806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:32.779460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:32.779750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:32.780034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:32.780894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] RBAC: Adding query filter params (role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6, user.id=349357072f3c4f3e8898b8580dc05bf9) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:32.781215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:32.782658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99803c-30b5-4198-b86c-266b7d8d3583 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:32.788038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 567/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:32 2022] GET /identity/v3/role_assignments?user.id=349357072f3c4f3e8898b8580dc05bf9&scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6&role.id=7805367da7144187ae1dd2e09705a569 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:34.758964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369adf-c8ad-42aa-8ba0-f144e9d29546 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:34.759496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369adf-c8ad-42aa-8ba0-f144e9d29546 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:34.759496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60369adf-c8ad-42aa-8ba0-f144e9d29546 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:34.760909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1118] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:34.765891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:34.766300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:34.766437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:34.783116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:34.784308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:34.849043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 568/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:34.871462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369adf-c8ad-42aa-8ba0-f144e9d29546 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:34.968139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60369adf-c8ad-42aa-8ba0-f144e9d29546 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:35.001108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:35.001934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:35.002216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:35.002488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] PATH_INFO: `/v3/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:35.003931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] RBAC: Authorizing `identity:get_role(role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:35.005326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:35.006658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:34 2022] GET /identity/v3/roles/7805367da7144187ae1dd2e09705a569 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:35.012255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:35.054461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:35.108965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b529926-e29c-44b9-af5c-98796846aa28 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:35.141897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:35.142714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:35.142997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:35.143276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] PATH_INFO: `/v3/users/349357072f3c4f3e8898b8580dc05bf9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:35.144633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] RBAC: Authorizing `identity:get_user(user_id=349357072f3c4f3e8898b8580dc05bf9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:35.146074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:35.147681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 569/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:35 2022] GET /identity/v3/users/349357072f3c4f3e8898b8580dc05bf9 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:35.153326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:35.163738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:35.249327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c8f2-9337-4134-950f-65cca5bcb34a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:35.280951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1c30f-afa0-41c1-ae9a-5648237e9a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:35.281782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1c30f-afa0-41c1-ae9a-5648237e9a43 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:35.282064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1c30f-afa0-41c1-ae9a-5648237e9a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:35.282339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1c30f-afa0-41c1-ae9a-5648237e9a43 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:35.283484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1c30f-afa0-41c1-ae9a-5648237e9a43 None admin] RBAC: Authorizing `identity:get_project(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:35.284927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1c30f-afa0-41c1-ae9a-5648237e9a43 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:35.286569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:35 2022] GET /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:35.292922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:35.389667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce61e80-a94c-474b-844b-5258062368f7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:35.400047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:35.423691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0688fb-ed37-43e3-95ff-299a9ae30070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:35.424612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0688fb-ed37-43e3-95ff-299a9ae30070 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:35.424899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0688fb-ed37-43e3-95ff-299a9ae30070 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:35.425179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0688fb-ed37-43e3-95ff-299a9ae30070 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6/users/349357072f3c4f3e8898b8580dc05bf9/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:35.427339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0688fb-ed37-43e3-95ff-299a9ae30070 None admin] RBAC: Authorizing `identity:create_grant(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6, user_id=349357072f3c4f3e8898b8580dc05bf9, role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:35.428820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0688fb-ed37-43e3-95ff-299a9ae30070 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:35.441842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 570/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:19:35 2022] PUT /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6/users/349357072f3c4f3e8898b8580dc05bf9/roles/7805367da7144187ae1dd2e09705a569 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:19:37.428851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9e825-fec7-4ce7-be74-f762c4707bbe None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:37.429405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9e825-fec7-4ce7-be74-f762c4707bbe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:37.429405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9e825-fec7-4ce7-be74-f762c4707bbe None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:37.430754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1124] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:37.435897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:37.436306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:37.436484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:37.453163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:37.454321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:37.523111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 571/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:37.546771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9e825-fec7-4ce7-be74-f762c4707bbe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:37.624447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:37.647866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9e825-fec7-4ce7-be74-f762c4707bbe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:37.681037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:37.681817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:37.682098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:37.682375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] PATH_INFO: `/v3/roles/7805367da7144187ae1dd2e09705a569` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:37.683823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] RBAC: Authorizing `identity:get_role(role_id=7805367da7144187ae1dd2e09705a569)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:37.685203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:37.686553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:37 2022] GET /identity/v3/roles/7805367da7144187ae1dd2e09705a569 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:37.692348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:37.786116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:37.789196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47a821-425b-406f-b249-22d724bfdc77 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:37.824024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:37.824920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:37.825201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:37.825479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] PATH_INFO: `/v3/users/349357072f3c4f3e8898b8580dc05bf9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:37.826761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] RBAC: Authorizing `identity:get_user(user_id=349357072f3c4f3e8898b8580dc05bf9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:37.828285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:37.829934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 572/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:37 2022] GET /identity/v3/users/349357072f3c4f3e8898b8580dc05bf9 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:37.835643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:37.930410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6558b8-c347-4931-a638-4a658dbcc992 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:37.963829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:37.964422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:37.964707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:37.964981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:37.978361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:38.030195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:38.031627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:38.124984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:38.130844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:38.227066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acdf14-e628-4fb8-b1e9-28a989839379 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:38.260115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac8ae2-a603-4d23-893d-962ab7340051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:38.261119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac8ae2-a603-4d23-893d-962ab7340051 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:38.261406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac8ae2-a603-4d23-893d-962ab7340051 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:38.261684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac8ae2-a603-4d23-893d-962ab7340051 None admin] PATH_INFO: `/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:38.262859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac8ae2-a603-4d23-893d-962ab7340051 None admin] RBAC: Authorizing `identity:get_project(project_id=eadb3e566bba4a2b8c3f0590b2afbbb6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:38.264319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac8ae2-a603-4d23-893d-962ab7340051 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:38.265969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 573/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:38 2022] GET /identity/v3/projects/eadb3e566bba4a2b8c3f0590b2afbbb6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:38.271769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:38.314866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:38.366554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301794ca-1b83-4b83-b39d-fbe76196542b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:38.398953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:38.399606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:38.400499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:38.400774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:38.401594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] RBAC: Adding query filter params (role.id=7805367da7144187ae1dd2e09705a569, scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6, user.id=349357072f3c4f3e8898b8580dc05bf9) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:38.401905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:38.403311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f246fd-82d1-4a9d-a54f-bd47cc6a6d05 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:38.408962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:38 2022] GET /identity/v3/role_assignments?user.id=349357072f3c4f3e8898b8580dc05bf9&scope.project.id=eadb3e566bba4a2b8c3f0590b2afbbb6&role.id=7805367da7144187ae1dd2e09705a569 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:40.462585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d905e3-eee5-477d-9273-80e2b6603adf None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:40.463161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d905e3-eee5-477d-9273-80e2b6603adf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:40.463161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d905e3-eee5-477d-9273-80e2b6603adf None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:40.464596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 574/1131] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:40.471415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:40.471862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:40.472332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:40.489833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:40.491029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:40.555248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:40.578338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d905e3-eee5-477d-9273-80e2b6603adf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:40.614035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:40.677952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d905e3-eee5-477d-9273-80e2b6603adf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:40.710976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c27292-223a-4d5f-8589-5ca8125f36a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:40.711754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c27292-223a-4d5f-8589-5ca8125f36a9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:40.712061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c27292-223a-4d5f-8589-5ca8125f36a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:40.712365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c27292-223a-4d5f-8589-5ca8125f36a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:40.713551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c27292-223a-4d5f-8589-5ca8125f36a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:40.714986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c27292-223a-4d5f-8589-5ca8125f36a9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:40.716607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 575/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:40.722391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:40.761214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:40.819186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0f3f5-2c03-487d-b80f-08b4110a1301 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:40.851600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f9258-1fb4-4274-8311-53408f66b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:40.852274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f9258-1fb4-4274-8311-53408f66b830 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:40.852565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f9258-1fb4-4274-8311-53408f66b830 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:40.853429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f9258-1fb4-4274-8311-53408f66b830 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:40.854143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f9258-1fb4-4274-8311-53408f66b830 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:40.855574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f9258-1fb4-4274-8311-53408f66b830 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:40.879306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:19:40 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:19:42.907941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989e465-8fb1-48ef-84ef-a97b9a4d6825 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:42.908475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989e465-8fb1-48ef-84ef-a97b9a4d6825 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:42.908475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989e465-8fb1-48ef-84ef-a97b9a4d6825 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:42.909810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 576/1135] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:42.915539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:42.915973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:42.916112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:42.933134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:42.934321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:42.998782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:43.022955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989e465-8fb1-48ef-84ef-a97b9a4d6825 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:43.121384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989e465-8fb1-48ef-84ef-a97b9a4d6825 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:43.123586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:43.154437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858993c-55bd-41c4-b822-c1a75a7581b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:43.155209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858993c-55bd-41c4-b822-c1a75a7581b4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:43.155489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858993c-55bd-41c4-b822-c1a75a7581b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:43.155762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858993c-55bd-41c4-b822-c1a75a7581b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:43.156935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858993c-55bd-41c4-b822-c1a75a7581b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:43.158406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858993c-55bd-41c4-b822-c1a75a7581b4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:43.159928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 577/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:19:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:43.165497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:43.186335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:43.263422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe0da-35de-4f27-b4a3-e3306d2838dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:43.295216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998d2fb-b64b-4fea-b25f-142f61385490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:43.295867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998d2fb-b64b-4fea-b25f-142f61385490 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:43.296150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998d2fb-b64b-4fea-b25f-142f61385490 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:43.296457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998d2fb-b64b-4fea-b25f-142f61385490 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:43.297153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998d2fb-b64b-4fea-b25f-142f61385490 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:43.298562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998d2fb-b64b-4fea-b25f-142f61385490 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:43.317561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:19:43 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:45.345495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46b340-d04f-4997-9c64-ddd744426a2c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:45.346024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46b340-d04f-4997-9c64-ddd744426a2c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:45.346024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46b340-d04f-4997-9c64-ddd744426a2c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:45.347408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 578/1139] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:45.352768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:45.353160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:45.353907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:45.370613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:45.371779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:45.437311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:45.460609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46b340-d04f-4997-9c64-ddd744426a2c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:45.558451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46b340-d04f-4997-9c64-ddd744426a2c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:45.590614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:45.591387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:45.591668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:45.591943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:45.601888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:45.609571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:45.610805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:45.612443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 579/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:45.616736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:45.625535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:45.714979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84350873-1ebe-4263-9c46-848ab67ce9a4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:45.748728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:45.749377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:45.749664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:45.749944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:45.750683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:45.750992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:45.752390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:45.760363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:45.766094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:45.797415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:45.863773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62114cbd-460c-465c-a3cd-82e91f65f5d9 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:45.896475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:45.897330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:45.897611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:45.897883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] PATH_INFO: `/v3/users/cb62ff00652f4d6c92773de0d8b913a7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:45.910585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] RBAC: Authorizing `identity:get_user(user_id=cb62ff00652f4d6c92773de0d8b913a7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:45.912028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:45.913674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 580/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:45 2022] GET /identity/v3/users/cb62ff00652f4d6c92773de0d8b913a7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:45.919944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:45.985557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:46.017965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b5fb5-8c38-48fe-a4ba-dd700680dbbd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:46.050730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:46.051368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:46.051657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:46.051937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:46.116903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:46.119059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:46.180288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:46.213870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:46.219720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:46.307413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:46.317155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262580b-dc7a-4654-9317-a6c605767e95 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:46.350052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f88e4-6ea1-42d5-a09e-0756a7f57b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:46.350900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f88e4-6ea1-42d5-a09e-0756a7f57b41 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:46.351186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f88e4-6ea1-42d5-a09e-0756a7f57b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:46.351467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f88e4-6ea1-42d5-a09e-0756a7f57b41 None admin] PATH_INFO: `/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:46.353263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f88e4-6ea1-42d5-a09e-0756a7f57b41 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad31d9e35749c0bf1554c904fa5cd9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:46.354701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f88e4-6ea1-42d5-a09e-0756a7f57b41 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:46.356398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 581/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:46 2022] GET /identity/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:46.362377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:46.457921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9b9ca-1798-41b8-87f4-0d680f0dcdd3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:46.491036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:46.491669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:46.491954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:46.492521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:46.493371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=94ad31d9e35749c0bf1554c904fa5cd9, user.id=cb62ff00652f4d6c92773de0d8b913a7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:46.493685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:46.495094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fabf43e-373c-4829-941f-72eaf77812e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:46.500361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:46 2022] GET /identity/v3/role_assignments?user.id=cb62ff00652f4d6c92773de0d8b913a7&scope.project.id=94ad31d9e35749c0bf1554c904fa5cd9&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:48.498682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72cc3-6828-45d7-afa7-60e14bf4e5f2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:48.499214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72cc3-6828-45d7-afa7-60e14bf4e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:48.499331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72cc3-6828-45d7-afa7-60e14bf4e5f2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:48.500675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 582/1147] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:48.505793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:48.506158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:48.506347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:48.522584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:48.523747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:48.549381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:48.589404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:48.612100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a72cc3-6828-45d7-afa7-60e14bf4e5f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:48.710429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a72cc3-6828-45d7-afa7-60e14bf4e5f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:48.743507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:48.744316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:48.744608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:48.744890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:48.754761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:48.762847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:48.763925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:48.769738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 583/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:19:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:48.770683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:48.871905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81cf114-350f-4423-818e-b860020ce3fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:48.904962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:48.905614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:48.905897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:48.906172 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:48.906909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:48.907213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:48.908595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:48.916394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:48.922394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:48.941871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:49.019784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e182946-0415-47fd-919b-abf3ab81a7e3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:49.051227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:49.052078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:49.052394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:49.052669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] PATH_INFO: `/v3/users/cb62ff00652f4d6c92773de0d8b913a7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:49.053823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] RBAC: Authorizing `identity:get_user(user_id=cb62ff00652f4d6c92773de0d8b913a7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:49.055250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:49.057390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 584/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:48 2022] GET /identity/v3/users/cb62ff00652f4d6c92773de0d8b913a7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:49.063311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:49.088548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:49.163117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68579269-1fe6-43f7-bd85-0610c9423747 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:49.195589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33db42-7b1a-4cf0-805f-cc516a9d757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:49.197018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33db42-7b1a-4cf0-805f-cc516a9d757d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:49.197305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33db42-7b1a-4cf0-805f-cc516a9d757d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:49.197583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33db42-7b1a-4cf0-805f-cc516a9d757d None admin] PATH_INFO: `/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:49.198754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33db42-7b1a-4cf0-805f-cc516a9d757d None admin] RBAC: Authorizing `identity:get_project(project_id=94ad31d9e35749c0bf1554c904fa5cd9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:49.200203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33db42-7b1a-4cf0-805f-cc516a9d757d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:49.201904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:49 2022] GET /identity/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:49.207975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:49.289535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:49.304161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d7061-587a-41d3-bed5-a25285b9eff3 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:49.337010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68aa4f-41d1-44d5-a2ee-90e4297ccfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:49.337879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68aa4f-41d1-44d5-a2ee-90e4297ccfb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:49.338162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68aa4f-41d1-44d5-a2ee-90e4297ccfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:49.338437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68aa4f-41d1-44d5-a2ee-90e4297ccfb3 None admin] PATH_INFO: `/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9/users/cb62ff00652f4d6c92773de0d8b913a7/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:49.340370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68aa4f-41d1-44d5-a2ee-90e4297ccfb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad31d9e35749c0bf1554c904fa5cd9, user_id=cb62ff00652f4d6c92773de0d8b913a7, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:49.341798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68aa4f-41d1-44d5-a2ee-90e4297ccfb3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:49.353523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 585/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:19:49 2022] PUT /identity/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9/users/cb62ff00652f4d6c92773de0d8b913a7/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:19:51.321348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74c1a-da5c-4b1b-ac20-906b3d77096d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:51.321894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74c1a-da5c-4b1b-ac20-906b3d77096d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:51.321985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74c1a-da5c-4b1b-ac20-906b3d77096d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:51.323321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1154] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:51.329214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:51.329633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:51.329787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:51.346150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:51.347320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:51.410537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 586/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:51.433883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74c1a-da5c-4b1b-ac20-906b3d77096d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:51.524199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:51.534038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74c1a-da5c-4b1b-ac20-906b3d77096d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:51.566051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:51.566823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:51.567107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:51.567380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:51.578116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:19:51.585545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:51.586697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:19:51.587734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:19:51.591660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:19:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:51.592726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:51.668897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:51.692736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a10109-35c8-4178-938b-79f5b3485b03 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:51.726323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:51.727006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:51.727305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:51.727593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:51.728380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:51.728693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:51.730066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:51.738021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 587/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:19:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:51.743986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:51.841420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bc411-b514-4672-852b-16fd32cabf24 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:51.874258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:51.875105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:51.875397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:51.875676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] PATH_INFO: `/v3/users/cb62ff00652f4d6c92773de0d8b913a7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:51.876891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] RBAC: Authorizing `identity:get_user(user_id=cb62ff00652f4d6c92773de0d8b913a7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:51.878313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:51.879905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:19:51 2022] GET /identity/v3/users/cb62ff00652f4d6c92773de0d8b913a7 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:51.885776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:51.981921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7f832-8272-4057-8134-537de3904523 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:52.015262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:52.015896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:52.016203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:52.016484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:52.025121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:52.081423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:52.082877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:52.178664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 588/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:52.184452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:52.205108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:52.280103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed25e99-2bc1-45ef-b9f7-ba1859a63a40 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:52.312340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457c40b-bfa4-42f0-8705-095e1e5b45c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:52.313169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457c40b-bfa4-42f0-8705-095e1e5b45c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:52.313449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457c40b-bfa4-42f0-8705-095e1e5b45c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:52.313722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457c40b-bfa4-42f0-8705-095e1e5b45c8 None admin] PATH_INFO: `/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:52.314869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457c40b-bfa4-42f0-8705-095e1e5b45c8 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad31d9e35749c0bf1554c904fa5cd9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:52.316313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457c40b-bfa4-42f0-8705-095e1e5b45c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:52.317973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:19:52 2022] GET /identity/v3/projects/94ad31d9e35749c0bf1554c904fa5cd9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:52.323943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:52.362648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:52.420166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92973-c077-45d0-bf7a-1d861f99d890 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:52.452695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:52.453283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:52.453566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:52.453843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:52.454661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=94ad31d9e35749c0bf1554c904fa5cd9, user.id=cb62ff00652f4d6c92773de0d8b913a7) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:52.454972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:52.456418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d136c-75e9-4103-b9c6-a577211bf512 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:52.462644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 589/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:19:52 2022] GET /identity/v3/role_assignments?user.id=cb62ff00652f4d6c92773de0d8b913a7&scope.project.id=94ad31d9e35749c0bf1554c904fa5cd9&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:54.545935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a64c0-8efc-4a4e-80d9-aa0162a6fba0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:54.546491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a64c0-8efc-4a4e-80d9-aa0162a6fba0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:54.546491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a64c0-8efc-4a4e-80d9-aa0162a6fba0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:54.547843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1162] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:54.554127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:54.554593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:54.554700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:54.572584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:54.573748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:54.638451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 590/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:54.660266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a64c0-8efc-4a4e-80d9-aa0162a6fba0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:54.746463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:54.758978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a64c0-8efc-4a4e-80d9-aa0162a6fba0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:54.791136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:54.791763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:54.792047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:54.792354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:54.855975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:54.857372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:54.939044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:54.951235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:19:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:19:54.957034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:54.986427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:55.051830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5c180-0bbd-474b-a05f-3f48dcbd4d14 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:55.084762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:55.085543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:55.085826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:55.086101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:55.092594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:55.094033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-039f7b67-9b6a-411c-8bf1-dba5f6ebf8f3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:19:55.107752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:19:55.108980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:19:55.110024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 591/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:19:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:19:55.114809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:55.210990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d513e7-0f4f-4c9e-8b3c-900aca92febb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:55.244296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:55.244934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:55.245216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:55.245491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:55.246193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:19:55.246498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:55.247842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781b5da-23a5-4780-b3de-c26e7eaae490 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:55.254705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Jul 19 12:19:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:19:57.238333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c36f9-79a0-48c5-8c2b-15acfa7c81e1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:57.238885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c36f9-79a0-48c5-8c2b-15acfa7c81e1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:57.238885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c36f9-79a0-48c5-8c2b-15acfa7c81e1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:57.240267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 592/1167] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:57.246268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1babf-6a45-4792-9d20-92e2cb460d17 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:57.246659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1babf-6a45-4792-9d20-92e2cb460d17 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:57.246840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1babf-6a45-4792-9d20-92e2cb460d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:57.263236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2ac1babf-6a45-4792-9d20-92e2cb460d17 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:57.264436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ac1babf-6a45-4792-9d20-92e2cb460d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:57.309408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:57.328521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:57.351142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c36f9-79a0-48c5-8c2b-15acfa7c81e1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:57.440901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:57.450466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c36f9-79a0-48c5-8c2b-15acfa7c81e1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:57.483297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c05568-24be-4333-8754-6f6e9526c6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:57.483932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c05568-24be-4333-8754-6f6e9526c6b5 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:57.484235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c05568-24be-4333-8754-6f6e9526c6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:57.484512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c05568-24be-4333-8754-6f6e9526c6b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:57.485234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c05568-24be-4333-8754-6f6e9526c6b5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:57.486590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c05568-24be-4333-8754-6f6e9526c6b5 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:57.500149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 593/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Tue Jul 19 12:19:57 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:19:59.529339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad73e94-859e-4c6b-b061-6405c67c8aa6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:59.529866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad73e94-859e-4c6b-b061-6405c67c8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:59.529981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad73e94-859e-4c6b-b061-6405c67c8aa6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:59.531283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1170] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:19:59.536498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:59.536932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:59.537063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:59.553955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:19:59.555173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:19:59.619861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 594/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:19:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:19:59.643647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad73e94-859e-4c6b-b061-6405c67c8aa6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:59.740291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad73e94-859e-4c6b-b061-6405c67c8aa6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:59.774500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f29d6c-4acd-4ee9-801d-5c53e653cb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:59.775344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29d6c-4acd-4ee9-801d-5c53e653cb84 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:59.775624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29d6c-4acd-4ee9-801d-5c53e653cb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:59.775895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f29d6c-4acd-4ee9-801d-5c53e653cb84 None admin] PATH_INFO: `/v3/domains/a76e1dad4b1f4069b36838f3a0d4f91d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:59.785936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f29d6c-4acd-4ee9-801d-5c53e653cb84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a76e1dad4b1f4069b36838f3a0d4f91d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:59.787307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f29d6c-4acd-4ee9-801d-5c53e653cb84 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:59.788946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Tue Jul 19 12:19:59 2022] GET /identity/v3/domains/a76e1dad4b1f4069b36838f3a0d4f91d => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:19:59.794779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:19:59.874532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:19:59.893220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fdcb7-bbd5-408c-8230-7ad7e56dbedc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:19:59.925921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4188b-807c-4100-a729-eb6c077b15a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:19:59.926574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4188b-807c-4100-a729-eb6c077b15a1 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:19:59.926861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4188b-807c-4100-a729-eb6c077b15a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:19:59.927140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4188b-807c-4100-a729-eb6c077b15a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:19:59.927855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4188b-807c-4100-a729-eb6c077b15a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:19:59.929326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4188b-807c-4100-a729-eb6c077b15a1 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:19:59.953666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 595/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:19:59 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:20:01.966363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09cb3b-a8cf-4464-9435-d3cd7f171f7f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:01.966859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09cb3b-a8cf-4464-9435-d3cd7f171f7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:01.966859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09cb3b-a8cf-4464-9435-d3cd7f171f7f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:01.968221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1174] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:20:01.973300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:01.973664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:01.973870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:01.990817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:20:01.991999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:20:02.056590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 596/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:20:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:20:02.080108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09cb3b-a8cf-4464-9435-d3cd7f171f7f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:02.124537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:02.179792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09cb3b-a8cf-4464-9435-d3cd7f171f7f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:02.215794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a79c9-6a71-4ebf-8341-25135b0cfe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:02.216734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a79c9-6a71-4ebf-8341-25135b0cfe3c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:02.216968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a79c9-6a71-4ebf-8341-25135b0cfe3c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:02.217241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a79c9-6a71-4ebf-8341-25135b0cfe3c None admin] PATH_INFO: `/v3/domains/a76e1dad4b1f4069b36838f3a0d4f91d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:02.218722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a79c9-6a71-4ebf-8341-25135b0cfe3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=a76e1dad4b1f4069b36838f3a0d4f91d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:02.220693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a79c9-6a71-4ebf-8341-25135b0cfe3c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:02.222402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Tue Jul 19 12:20:02 2022] GET /identity/v3/domains/a76e1dad4b1f4069b36838f3a0d4f91d => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:02.230067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:02.290313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:02.326569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5b412-15c4-4f90-82eb-ab32ee284fd1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:02.359114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf12c3-5be6-4070-a265-836ea85c3408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:02.359781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf12c3-5be6-4070-a265-836ea85c3408 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:02.360064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf12c3-5be6-4070-a265-836ea85c3408 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:02.360368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf12c3-5be6-4070-a265-836ea85c3408 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:02.361070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf12c3-5be6-4070-a265-836ea85c3408 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:02.362508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf12c3-5be6-4070-a265-836ea85c3408 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:02.381421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 597/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:20:02 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:20:04.451672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae2cf-3568-481f-a7d8-0232d2f8d523 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:04.452277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae2cf-3568-481f-a7d8-0232d2f8d523 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:04.452277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ae2cf-3568-481f-a7d8-0232d2f8d523 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:04.453624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1178] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:20:04.458891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:04.459305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:04.459457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:04.476460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:20:04.477638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:20:04.541959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 598/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:20:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:20:04.565097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ae2cf-3568-481f-a7d8-0232d2f8d523 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:04.668345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ae2cf-3568-481f-a7d8-0232d2f8d523 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:04.678162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:04.701544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:04.702370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:04.702611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:04.702889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:04.713717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:20:04.721248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:20:04.722415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:20:04.723783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:20:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:20:04.728496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:04.777054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:04.826175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62df19a-34db-4d51-9184-21cbd0f759ab None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:04.858522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:04.859180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:04.859471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:04.859805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:04.860588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:20:04.860899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:04.862277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:04.870599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 599/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:20:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:04.876347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:04.971912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9041f-7c4f-4eb2-8a7a-9b415fda394f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:05.004479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:05.005335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:05.005621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:05.005899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] PATH_INFO: `/v3/users/f0427b37029b4e1ea47f1280ad6aec01` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:05.019223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] RBAC: Authorizing `identity:get_user(user_id=f0427b37029b4e1ea47f1280ad6aec01)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:05.020694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:05.022316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:20:04 2022] GET /identity/v3/users/f0427b37029b4e1ea47f1280ad6aec01 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:05.027900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:05.124747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114401f3-4a2b-40ad-9727-5b720934f2c4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:05.137166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:05.157587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:05.158219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:05.158503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:05.158782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:05.223541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:05.224949 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:05.318885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 600/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:20:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:20:05.325158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:05.354437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:05.422880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c2f2c-deb7-4c1c-96d5-1250da2ee051 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:05.455043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011c3d0-4c9b-43f3-8bbb-cf2c8d2280cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:05.455886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c3d0-4c9b-43f3-8bbb-cf2c8d2280cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:05.456167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c3d0-4c9b-43f3-8bbb-cf2c8d2280cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:05.456471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011c3d0-4c9b-43f3-8bbb-cf2c8d2280cc None admin] PATH_INFO: `/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:05.457630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c3d0-4c9b-43f3-8bbb-cf2c8d2280cc None admin] RBAC: Authorizing `identity:get_project(project_id=83f4d17d9ba84d2b90a643e0fc4922d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:05.459055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011c3d0-4c9b-43f3-8bbb-cf2c8d2280cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:05.460792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:20:05 2022] GET /identity/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:05.467426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:05.481206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:05.562192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1e84b-8acc-4366-810d-ff2d42c4fbbe None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:05.594366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:05.595003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:05.595285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:05.595558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:05.596399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=83f4d17d9ba84d2b90a643e0fc4922d5, user.id=f0427b37029b4e1ea47f1280ad6aec01) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:20:05.596709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:05.598118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67025-a785-4ae7-a8d1-145725dc68da None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:05.603369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 601/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:20:05 2022] GET /identity/v3/role_assignments?user.id=f0427b37029b4e1ea47f1280ad6aec01&scope.project.id=83f4d17d9ba84d2b90a643e0fc4922d5&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:07.612685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bb358-6889-4e75-ba6f-493d23386d37 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:07.613120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bb358-6889-4e75-ba6f-493d23386d37 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:07.613246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bb358-6889-4e75-ba6f-493d23386d37 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:07.614633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1186] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:20:07.620103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:07.620539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:07.620703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:07.637580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:20:07.638763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:20:07.703427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 602/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:20:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:20:07.726495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bb358-6889-4e75-ba6f-493d23386d37 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:07.818059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:07.825535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bb358-6889-4e75-ba6f-493d23386d37 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:07.858586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:07.859350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:07.859633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:07.859910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:07.871944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:20:07.879474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:20:07.880621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:20:07.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:20:07.886705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:07.894731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:07.984678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537993d-bb54-46aa-beb1-c15b1ec762cd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:08.017047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:08.017702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:08.017984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:08.018259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:08.018995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:20:08.019299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:08.020698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:08.029095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 603/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:20:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:08.035030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:08.131568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e757e59-d231-4cb2-a409-5b89672cde5c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:08.164632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:08.165478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:08.165764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:08.166043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] PATH_INFO: `/v3/users/f0427b37029b4e1ea47f1280ad6aec01` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:08.167227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] RBAC: Authorizing `identity:get_user(user_id=f0427b37029b4e1ea47f1280ad6aec01)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:08.168696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:08.170311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:20:08 2022] GET /identity/v3/users/f0427b37029b4e1ea47f1280ad6aec01 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:08.175983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:08.239803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:08.272471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05433c10-ed98-4a55-a946-f98d47bbe89d None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:08.305654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f52a43-b457-4132-967d-2fce12dc48fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:08.306514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f52a43-b457-4132-967d-2fce12dc48fd None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:08.306800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f52a43-b457-4132-967d-2fce12dc48fd None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:08.307073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f52a43-b457-4132-967d-2fce12dc48fd None admin] PATH_INFO: `/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:08.308281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f52a43-b457-4132-967d-2fce12dc48fd None admin] RBAC: Authorizing `identity:get_project(project_id=83f4d17d9ba84d2b90a643e0fc4922d5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:08.309722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f52a43-b457-4132-967d-2fce12dc48fd None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:08.311412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 604/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:20:08 2022] GET /identity/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:08.317368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:08.342195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:08.413006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456e42c-05bd-456e-961d-890cb119021f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:08.445337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b95dcd-e7a9-486d-84e8-c51bc1dd7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:08.446204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b95dcd-e7a9-486d-84e8-c51bc1dd7431 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:08.446487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b95dcd-e7a9-486d-84e8-c51bc1dd7431 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:08.446763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b95dcd-e7a9-486d-84e8-c51bc1dd7431 None admin] PATH_INFO: `/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5/users/f0427b37029b4e1ea47f1280ad6aec01/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:08.448695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b95dcd-e7a9-486d-84e8-c51bc1dd7431 None admin] RBAC: Authorizing `identity:create_grant(project_id=83f4d17d9ba84d2b90a643e0fc4922d5, user_id=f0427b37029b4e1ea47f1280ad6aec01, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:08.450110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b95dcd-e7a9-486d-84e8-c51bc1dd7431 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:08.462041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:20:08 2022] PUT /identity/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5/users/f0427b37029b4e1ea47f1280ad6aec01/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:20:10.447368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18e53a-5526-45c2-9817-879acc89c776 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:10.447913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18e53a-5526-45c2-9817-879acc89c776 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:10.447913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18e53a-5526-45c2-9817-879acc89c776 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:10.449314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 605/1193] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:20:10.454433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c426e-27ae-452f-9734-2276fea05261 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:10.454795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c426e-27ae-452f-9734-2276fea05261 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:10.454983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c426e-27ae-452f-9734-2276fea05261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:10.471191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-426c426e-27ae-452f-9734-2276fea05261 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:20:10.472387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-426c426e-27ae-452f-9734-2276fea05261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:20:10.535729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:20:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:20:10.558820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18e53a-5526-45c2-9817-879acc89c776 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:10.659922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18e53a-5526-45c2-9817-879acc89c776 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:10.692827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:10.693607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:10.693887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:10.694163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:10.696777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:10.705096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:20:10.712733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:20:10.713916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:20:10.714962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 606/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:20:10.720050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c426e-27ae-452f-9734-2276fea05261 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:10.735011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:10.816083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c426e-27ae-452f-9734-2276fea05261 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:10.848345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:10.848990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:10.849274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:10.849549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:10.850286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:20:10.850589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:10.852018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:10.859730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:10.865308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:10.962309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4130b9-bc3d-4c51-97ce-d9764495c0b0 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:10.994980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:10.995842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:10.996125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:10.996430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] PATH_INFO: `/v3/users/f0427b37029b4e1ea47f1280ad6aec01` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:10.997601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] RBAC: Authorizing `identity:get_user(user_id=f0427b37029b4e1ea47f1280ad6aec01)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:10.999046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:11.000696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 607/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Jul 19 12:20:10 2022] GET /identity/v3/users/f0427b37029b4e1ea47f1280ad6aec01 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:11.006399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:11.080816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:11.102698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479f054-9597-4b84-8027-599e8b5e4665 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:11.135464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:11.136087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:11.136402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:11.136681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:11.200632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:11.202066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:11.278117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:11.300743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:20:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:20:11.306581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:11.366566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:11.403722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96f14-5bcb-45c7-b854-bc292523e16e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:11.435908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bb3dd-ba19-4463-af8f-8320dadc9410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:11.436786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bb3dd-ba19-4463-af8f-8320dadc9410 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:11.437070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bb3dd-ba19-4463-af8f-8320dadc9410 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:11.437343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bb3dd-ba19-4463-af8f-8320dadc9410 None admin] PATH_INFO: `/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:11.438501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bb3dd-ba19-4463-af8f-8320dadc9410 None admin] RBAC: Authorizing `identity:get_project(project_id=83f4d17d9ba84d2b90a643e0fc4922d5)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:11.439932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bb3dd-ba19-4463-af8f-8320dadc9410 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:11.441606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 608/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Jul 19 12:20:11 2022] GET /identity/v3/projects/83f4d17d9ba84d2b90a643e0fc4922d5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:11.448084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:20:11.543814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe73b-537f-4b94-8737-e2baed75107b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:20:11.577010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:20:11.577653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:20:11.577939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:20:11.578217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:20:11.579043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=83f4d17d9ba84d2b90a643e0fc4922d5, user.id=f0427b37029b4e1ea47f1280ad6aec01) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:20:11.579362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:20:11.580828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda97a04-eb7a-48a9-a7d8-8c1bd5259929 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:20:11.586544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:20:11 2022] GET /identity/v3/role_assignments?user.id=f0427b37029b4e1ea47f1280ad6aec01&scope.project.id=83f4d17d9ba84d2b90a643e0fc4922d5&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:20:21.431877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=34 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:20:21.583897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=80 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:06.783678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308359da-8045-4fea-bb44-394e0dd08623 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:06.783678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308359da-8045-4fea-bb44-394e0dd08623 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:06.783678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308359da-8045-4fea-bb44-394e0dd08623 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:06.785102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 609/1201] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:06.796047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:06.796581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:06.796670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:06.817887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:06.819099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:06.907301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:06.934200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308359da-8045-4fea-bb44-394e0dd08623 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:07.058537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308359da-8045-4fea-bb44-394e0dd08623 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:07.097812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:07.099175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:07.099462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:07.099773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:07.114816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:07.116606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:07.136121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:07.137181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:07.138155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:07.138155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:07.138155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:07.138155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:07.138155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 610/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:07.144054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:07.282111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483b026-7b22-4b87-87bd-acda55ac3dfd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:07.323112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:07.330582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:07.331718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:07.332065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:07.332409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:07.333808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:07.334150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:07.335934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e53b1-ad6f-426c-9b5e-bd72a35a0462 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:07.353534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:21:07.354810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:07.423901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:07.485385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf05576-952e-4b21-8900-a13389018eca None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:07.536979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d820bc-138e-4246-ab3b-f255299d167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:07.537636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d820bc-138e-4246-ab3b-f255299d167e None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:07.537966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d820bc-138e-4246-ab3b-f255299d167e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:07.538209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d820bc-138e-4246-ab3b-f255299d167e None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:07.539328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d820bc-138e-4246-ab3b-f255299d167e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:07.545396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d820bc-138e-4246-ab3b-f255299d167e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:07.569090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 611/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Jul 19 12:21:07 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:21:09.831252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223870c-a980-41e5-b7fa-0b143723994f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:09.831833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223870c-a980-41e5-b7fa-0b143723994f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:09.831833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223870c-a980-41e5-b7fa-0b143723994f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:09.834035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1206] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:09.842189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:09.842189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:09.842189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:09.863521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:09.864738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:09.944761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 612/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:09.976552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223870c-a980-41e5-b7fa-0b143723994f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:10.107721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223870c-a980-41e5-b7fa-0b143723994f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:10.146418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:10.147123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:10.147415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:10.147695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:10.159720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:21:10.168291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:10.169459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:10.173439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:21:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:10.175800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:10.223903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:10.301336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40677f3-a4c6-4937-81e7-43795cfac21f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:10.342289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:10.342289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:10.342289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:10.342289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:10.342708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:10.342708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:10.343343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:10.352817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 613/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:21:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:10.359961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:10.414706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:10.477206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde5a62-6ff1-4d90-a901-39cc3dcf2baa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:10.516334 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:10.517175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:10.517459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:10.517734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:10.529392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:10.531030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:10.556278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:10.557282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:10.563056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:10.563056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:10.563056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:10.563056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:10.563056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:21:10.565087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:10.689057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:10.692586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1853768-e128-464d-80ea-0417bd6cf3c7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:10.732545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:10.733214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:10.733498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:10.733770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:10.734777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:10.735088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:10.736571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:10.750273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 614/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:10.757259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:10.883324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130b79d-5496-4730-9da8-6bf2a31b38c1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:10.884974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:10.923108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:10.923906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:10.924209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:10.924484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:10.933383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:10.935252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:21:10.953377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:10.954669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:10.955659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:10.960479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:10.964029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:11.078892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fb106-d0f9-4508-bb8f-d2abb667726e None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:11.123293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:11.124082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:11.124454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:11.124806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:11.125789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:11.126213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:11.127645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:11.147937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 615/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jul 19 12:21:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:11.154112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:11.287694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0f691-c0fc-4d2e-82ab-e0478b49b700 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:11.326656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:11.327318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:11.327593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:11.327868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:11.362841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:11.407386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:11.408898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:11.526263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:21:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:21:11.527528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:11.546823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:11.635712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07aaa74-3362-4eac-b4b2-013710831374 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:11.685953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:11.687009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:11.687358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:11.687752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:11.701106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:11.703100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:11.725156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:11.726369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:11.727681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:11.727681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:11.727681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:11.727681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:11.729438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 616/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:11.735784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:11.807012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:11.847881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b063a-a538-4915-8d27-7e391a3fbd13 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:11.889577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:11.890361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:11.890555 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:11.890836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:11.891562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:11.891891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:11.893368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:11.906912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:11.913434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:12.001385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:12.042964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf821-8d78-4d10-87b1-6c2301d3ba18 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:12.095070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:12.095812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:12.096095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:12.096407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:12.104770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:12.106260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:21:12.121917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:12.123072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:12.124042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:12.128483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 617/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:21:12.129975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:12.254723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949ae99-ce73-42d8-98db-2bffbf84fbd1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:12.281374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:12.297674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:12.298342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:12.298734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:12.299042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:12.299970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:12.300358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:12.302400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4de8008-5c87-4d96-9dcd-f57dac43dda8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:12.320871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:12.322451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:12.441699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa0d96-b910-498c-bff7-6c34adc829b4 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:12.476505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:12.483086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:12.483824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:12.484568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:12.485263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:12.486293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=06d3500f0bda4ec58b2d561f13eb42d9) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:12.486719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:12.488734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d60f20-55e9-4c11-ad31-a8c5e7411b90 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:12.496022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 618/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:21:12 2022] GET /identity/v3/role_assignments?user.id=06d3500f0bda4ec58b2d561f13eb42d9&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:14.752692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd44aa-73fd-461e-8d73-a03d0ca18d4f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:14.753254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd44aa-73fd-461e-8d73-a03d0ca18d4f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:14.753348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd44aa-73fd-461e-8d73-a03d0ca18d4f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:14.755267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1220] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:14.760892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:14.761344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:14.761455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:14.778005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:14.779171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:14.848135 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 619/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:14.871382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd44aa-73fd-461e-8d73-a03d0ca18d4f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:14.980361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd44aa-73fd-461e-8d73-a03d0ca18d4f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:15.014947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:15.015898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:15.016272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:15.016629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:15.027268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:21:15.036711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:15.037853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:15.038753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:21:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:15.044573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:15.144085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e2a8d-1fef-494e-a48a-9fdcadf725b4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:15.172466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:15.184565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:15.185316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:15.185665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:15.185941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:15.186688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] RBAC: Adding query filter params (name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:15.186995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:15.188403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:15.196925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 620/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:21:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:21:15.202766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:15.248374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:15.311996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314d9fb-1a82-4124-974c-7750e3b64faf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:15.351252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:15.352136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:15.352455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:15.352833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:15.367358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:15.369213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:15.388843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:15.389939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:15.390926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:15.390926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:15.390926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:15.390926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:15.390926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:21:15.397153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:15.517666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f8ff-13c3-4bf9-aa12-9fe82e1b078c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:15.553321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:15.553987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:15.554268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:15.554544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:15.555264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:15.555569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:15.557029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:15.570086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 621/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:15.576000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:15.679965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:15.687636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab50afb-e469-4921-a1a4-055c3e1dbe63 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:15.725601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:15.726431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:15.726758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:15.726997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:15.734836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:15.736308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:21:15.752068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:15.754125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:15.755366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:21:15.759878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:15.814994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:15.876024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cb056-4389-4e80-a5b0-841ff76260eb None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:15.913152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:15.914186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:15.914842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:15.915126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:15.916116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:15.916616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:15.918191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:15.937564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 622/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jul 19 12:21:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:15.944147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:16.052441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6da76-994a-4ce0-a8c2-a9d706f9e3f8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:16.089669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:16.090673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:16.090933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:16.091296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:16.103678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:16.105169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:16.122207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:16.123350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:16.124290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:16.124290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:16.124290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:16.124290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:16.124290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:16.129349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:16.246644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33988-e829-45f2-9003-80bb5d73002c None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:16.248998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:16.283683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:16.284548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:16.284874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:16.285155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:16.286434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:16.286773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:16.288297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:16.306497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 623/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:21:16.307755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:16.320648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:16.451302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379e31-d05e-4c48-88d9-741ec931076e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:16.500785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:16.501497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:16.501780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:16.502058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:16.512060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:16.514452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:21:16.535580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:16.536744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:16.537814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:16.538356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:21:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:16.543952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:16.674166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5ab78-4c1b-4e03-bd5d-e820c2cd5083 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:16.710771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:16.711428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:16.711709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:16.711982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:16.712842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:16.713104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:16.714546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84fe3-d20c-4a44-af37-40ffab59020e None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:16.728132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 624/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:21:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:16.734135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:16.804478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:16.840403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bdc7e-81ce-45aa-9d31-dc60d4d5a87b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:16.877830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0156c9d-f974-4687-bb8e-09df97c70425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:16.878714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156c9d-f974-4687-bb8e-09df97c70425 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:16.879677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156c9d-f974-4687-bb8e-09df97c70425 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:16.879899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0156c9d-f974-4687-bb8e-09df97c70425 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/06d3500f0bda4ec58b2d561f13eb42d9/roles/f05e6e768b1342638eedbbaeab452f65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:16.894289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156c9d-f974-4687-bb8e-09df97c70425 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=06d3500f0bda4ec58b2d561f13eb42d9, role_id=f05e6e768b1342638eedbbaeab452f65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:16.895748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0156c9d-f974-4687-bb8e-09df97c70425 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:16.924209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:21:16 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/06d3500f0bda4ec58b2d561f13eb42d9/roles/f05e6e768b1342638eedbbaeab452f65 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:21:19.005644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba154f-b08d-46ec-a03d-026d9f64c7bf None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:19.006272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba154f-b08d-46ec-a03d-026d9f64c7bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:19.006272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba154f-b08d-46ec-a03d-026d9f64c7bf None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:19.007745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 625/1233] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:19.013655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:19.014071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:19.014245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:19.031323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:19.032540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:19.112418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:19.138629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba154f-b08d-46ec-a03d-026d9f64c7bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:19.166676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:19.251385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba154f-b08d-46ec-a03d-026d9f64c7bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:19.288975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:19.289752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:19.290035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:19.290309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:19.302177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:21:19.310344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:19.311497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:19.312681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 626/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Jul 19 12:21:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:19.318037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:19.380453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:19.444424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9df9a9-6c2c-4483-a89f-85ef33495379 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:19.482175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:19.482839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:19.483125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:19.483409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:19.484153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:19.484490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:19.485870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:19.499932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Jul 19 12:21:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:19.501371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:19.595558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:19.615455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d878c4-8e94-45eb-8033-f750a0611a7a None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:19.654721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:19.655531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:19.655818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:19.656095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:19.668825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:19.670325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:19.688630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:19.689624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:19.690541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:19.690541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:19.690541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:19.690541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:19.690541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 627/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:19.696364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:19.806961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c1384-a90c-4581-8cfe-5ea307e61144 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:19.818412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:19.846410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:19.847065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:19.847417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:19.847779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:19.848606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:19.848915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:19.850350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:19.862880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:19.869561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:19.976501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be0e3-8c21-43d1-a6fb-5577d9c6d5d6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:20.015974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:20.016883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:20.017171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:20.017446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:20.020395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:20.026023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:20.027842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:21:20.042143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:20.043649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:20.044936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 628/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:20.049321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:20.157692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481e6b-0403-49f0-9c8d-b19c0e062c6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:20.194490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:20.195221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:20.195432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:20.195706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:20.196577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:20.196889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:20.198323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:20.218258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jul 19 12:21:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:20.224314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:20.328274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671f26b-fe8c-459a-8713-26f014d5ba2f None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:20.370674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:20.371558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:20.372013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:20.372423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:20.445218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:20.452273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:20.453736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:20.558433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 629/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:21:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:21:20.563994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:20.595234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:20.676745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e6106-0ba2-4a17-9736-36a579a9e24c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:20.711841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:20.712590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:20.712878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:20.713156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:20.724050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:20.725528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:20.744530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:20.745901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:20.747241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:20.747241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:20.747241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:20.747241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:20.747241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:20.751468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:20.856142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:20.864166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bda7f-14f1-4f2a-9485-c1afb29bb8db None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:20.905521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:20.906189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:20.906475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:20.906754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:20.907473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:20.907785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:20.909196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:20.921070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 630/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:20.927293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:21.021744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:21.047072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df9270-3c5a-43ec-b53f-efdd4d6eb319 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:21.086130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:21.086927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:21.087210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:21.087486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:21.095245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:21.096979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:21:21.112572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:21.114064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:21.114962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:21:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:21.120836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:21.233004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d3d88-0749-4a18-aabc-62aefd836b61 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:21.271703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:21.272506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:21.272776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:21.273064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:21.273974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:21.274329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:21.275825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c1469-3d76-4367-aff8-aa6ecff1e520 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:21.290741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 631/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:21:21.296949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:21.419845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35741814-b439-4e1b-a012-4dfd6f40137d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:21.453820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:21.454444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:21.454720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:21.454995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:21.455813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] RBAC: Adding query filter params (role.id=f05e6e768b1342638eedbbaeab452f65, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=06d3500f0bda4ec58b2d561f13eb42d9) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:21.456129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:21.458110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647e5b7-97f2-483e-babd-8d6df7359456 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:21.464071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:21:21 2022] GET /identity/v3/role_assignments?user.id=06d3500f0bda4ec58b2d561f13eb42d9&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=f05e6e768b1342638eedbbaeab452f65 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:23.737372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4fb9-2232-4a1f-9a2c-a0bde1a7a3f0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:23.737908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4fb9-2232-4a1f-9a2c-a0bde1a7a3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:23.738121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e4fb9-2232-4a1f-9a2c-a0bde1a7a3f0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:23.739980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 632/1247] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:23.745711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:23.746345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:23.746345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:23.752702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:23.765747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:23.766944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:23.845119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:23.876474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4fb9-2232-4a1f-9a2c-a0bde1a7a3f0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:23.899745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:23.998846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e4fb9-2232-4a1f-9a2c-a0bde1a7a3f0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:24.039815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:24.040736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:24.041026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:24.041301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:24.052437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:21:24.060478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:24.062290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:24.063743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 633/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:21:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:24.068992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:24.180348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164de11-d4e1-4371-aaa1-0c244121a667 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:24.213993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:24.219516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:24.220206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:24.220491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:24.220805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:24.221550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:24.221864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:24.224330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:24.233608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:21:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:24.240083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:24.323354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:24.351258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6020a-8eab-43fe-bb9d-da84b35a1241 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:24.401037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:24.402138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:24.402541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:24.402814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:24.416043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:24.417542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:24.439817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:24.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:24.442703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:24.442703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:24.442703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:24.442703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:24.442703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 634/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:24.448120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:24.564553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c59a5-71ca-464e-97a1-7f14b6165100 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:24.604255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:24.605001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:24.605338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:24.605690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:24.606547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:24.606887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:24.608514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:24.623134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:24.630170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:24.743546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0e7ad-7e6b-481a-ba89-52426a7c68e7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:24.770767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:24.779992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:24.780904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:24.781186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:24.781460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:24.789586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:24.791018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:21:24.808214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:24.809326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:24.810455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 635/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 19 12:21:24.816012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:24.881018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:24.941933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7934549-bd48-4b4f-977d-2ebe0b9db924 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:24.981941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:24.982678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:24.983027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:24.983343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:24.984431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:24.984743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:24.986355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:25.009497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jul 19 12:21:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:21:25.010803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:25.135282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f2ac2-660a-43b2-990b-3e224562e379 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:25.171229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:25.171919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:25.172321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:25.172553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:25.231732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:25.245745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:25.247430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:25.364451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 636/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:21:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:21:25.370900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:25.493961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d7d1b-6a77-4c9f-b9a9-39916cb67b58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:25.508050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:25.530959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:25.531793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:25.532085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:25.532399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:25.544022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:25.545513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:25.565493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:25.567000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:25.567910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:25.567910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:25.567910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:25.567910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:25.567910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:25.573370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:25.672608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:25.685544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f007f-2602-4131-a196-aaae4cde65ed None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:25.722032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:25.722745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:25.723027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:25.723307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:25.724046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:25.724381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:25.725796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:25.738870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 637/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:25.745709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:25.861771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3324-8822-451f-a11a-eee74447c3ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:25.896803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:25.897601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:25.897883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:25.898155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:25.905494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:25.906952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:21:25.921406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:25.922497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:25.923595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:25.929352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:26.032842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07bb6-28d3-4dfb-a865-58a391752470 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:26.073912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:26.074651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:26.074956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:26.075235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:26.076323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:26.076643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:26.078154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf73bb-2530-4b27-a949-0d69c6ef4ea9 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:26.094958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 638/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:26.102033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:26.130361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:26.207659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c61169-dafd-4629-96d4-7c222bc33e9f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:26.242380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:26.243033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:26.243315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:26.243591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:26.244451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=06d3500f0bda4ec58b2d561f13eb42d9) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:26.244762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:26.246189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0e9b3-beb6-49cc-9d15-00113b0767e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:26.252113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:21:26 2022] GET /identity/v3/role_assignments?user.id=06d3500f0bda4ec58b2d561f13eb42d9&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=ac904f211a8f450fbf556c879296af4b => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:28.284167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10199e6c-8443-4c0b-a902-95cd7b16d434 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:28.284758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10199e6c-8443-4c0b-a902-95cd7b16d434 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:28.284853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10199e6c-8443-4c0b-a902-95cd7b16d434 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:28.286184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 639/1261] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:28.291260 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:28.291694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:28.291826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:28.308731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:28.309921 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:28.383768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:28.409161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10199e6c-8443-4c0b-a902-95cd7b16d434 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:28.425072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:28.518387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10199e6c-8443-4c0b-a902-95cd7b16d434 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:28.552167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:28.552918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:28.553204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:28.553482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:28.563485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:21:28.571097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:28.572294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:28.573322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 640/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:21:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:28.578259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:28.596025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:28.675876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd881d-9655-44e5-84e0-6505ec3e85dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:28.707820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:28.708503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:28.708786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:28.709063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:28.709799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:28.710104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:28.711453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:28.719330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:21:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:28.725687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:28.797814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:28.830409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4160a2a-5de4-4dbc-a8a0-29f0fb86ebff None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:28.863332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:28.864114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:28.864432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:28.864713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:28.875232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:28.876723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:28.893306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:28.894850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:28.895974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:28.895974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:28.895974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:28.895974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:28.895974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 641/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:28.900858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:28.978716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:28.997280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e654-fa5a-4995-8669-33c84a5cc0b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.029369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.030009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.030293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.030566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.031267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:29.031571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.032948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.044738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:29.050204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:29.146955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ae4e4-2a2b-4e8a-b587-9739b32c3543 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.165866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:29.180013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.180825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.181107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.181381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.188568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.190003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:21:29.203791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:29.204941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:29.206069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 642/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:29.210712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:29.307935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396579fd-44cc-48ab-8d6d-19c0c7bd92dc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.341734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.342494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.342850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.343127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.344114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:29.344516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.346112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.357600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:29.369074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jul 19 12:21:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:29.374918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:29.472843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1e619-3686-4382-8cd9-6ba295d771a6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.504750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.505520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.505800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.506071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.516122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.517584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:29.534386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:29.535465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:29.536606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:29.536606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:29.536606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:29.536606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:29.536606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 643/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:29.541139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:29.638735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357c9f5-2bda-48b8-be6f-f25fa290b314 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.670728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.671376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.671661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.671938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.672683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:29.672994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.674355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.685736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:29.691804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:29.722290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:29.789274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d7460-1c08-4863-a7d9-47b446e9db22 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.821950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.822717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.822997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.823270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.829697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.831125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:21:29.844872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:29.845980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:29.847084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 644/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:21:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:29.853400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:29.910165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:29.949704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e93cb-39ab-4818-9ef3-9a986fc3db80 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:29.981382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:29.982019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:29.982301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:29.982577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:29.983344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:29.983651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:29.985085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d4bcb-fed9-4be6-8a4d-2b3bc5e42c6c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:29.997330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:21:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:30.003129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:30.090162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:30.099121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61f28f-daad-4fb8-84f8-b4886f6243e7 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:30.132683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd697d-f330-4fcc-97fc-d4a2517dbed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:30.133555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd697d-f330-4fcc-97fc-d4a2517dbed3 None admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:30.133840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd697d-f330-4fcc-97fc-d4a2517dbed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:30.134116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd697d-f330-4fcc-97fc-d4a2517dbed3 None admin] PATH_INFO: `/v3/projects/b1a666f3a1a144858853d303838a22b9/users/06d3500f0bda4ec58b2d561f13eb42d9/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:30.136029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd697d-f330-4fcc-97fc-d4a2517dbed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b1a666f3a1a144858853d303838a22b9, user_id=06d3500f0bda4ec58b2d561f13eb42d9, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:30.137488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd697d-f330-4fcc-97fc-d4a2517dbed3 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:30.149686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 645/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Jul 19 12:21:30 2022] PUT /identity/v3/projects/b1a666f3a1a144858853d303838a22b9/users/06d3500f0bda4ec58b2d561f13eb42d9/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:21:32.135956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20779cb-fcf0-4da6-a66a-1f9d5a1a5c35 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:32.136517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20779cb-fcf0-4da6-a66a-1f9d5a1a5c35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:32.136517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20779cb-fcf0-4da6-a66a-1f9d5a1a5c35 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:32.137902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1274] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:32.142990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:32.143357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:32.143553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:32.160347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:32.161589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:32.226753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 646/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:32.250541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20779cb-fcf0-4da6-a66a-1f9d5a1a5c35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:32.350101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20779cb-fcf0-4da6-a66a-1f9d5a1a5c35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:32.369647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:32.386223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:32.386946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:32.387233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:32.387512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:32.397387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 19 12:21:32.405207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:32.406308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:32.412663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Jul 19 12:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:32.413515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:32.490276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:32.508357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18440cb0-4b7c-4d12-b2d7-660b32f14d22 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:32.541083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:32.541732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:32.542026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:32.542291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:32.543025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:32.543327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:32.544721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:32.552457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 647/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Jul 19 12:21:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:32.557971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:32.656759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612d7aa-f2ae-4745-8cc2-b8fcc0ba8e99 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:32.688665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:32.689427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:32.689708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:32.689977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:32.701006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:32.702404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:32.719521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:32.720636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:32.721599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:32.721599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:32.721599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:32.721599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:32.721599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:32.726961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:32.824087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9be63b-e077-450b-a777-1d2e03e74e1b None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:32.856955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:32.857594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:32.857874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:32.858146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:32.858851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:32.859152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:32.860530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:32.872146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 648/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:32.877761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:32.908599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:32.975913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c335acb-426f-4a93-871f-d9a694030144 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:33.008298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:33.009077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:33.009364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:33.009640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:33.016780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:33.018202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 19 12:21:33.032151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:33.033277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:33.034912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:33.039469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:33.043110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:33.134403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058cfd6-6a60-4136-974a-bae2f1f575ca None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:33.167036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:33.167642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:33.167925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:33.168227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:33.169055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:33.169367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:33.170791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:33.188132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 649/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Jul 19 12:21:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:33.193956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:33.280506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:33.291086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f0c5-f62a-4389-8c80-4093c30666f8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:33.322953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:33.323579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:33.323865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:33.324140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:33.400084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:33.401582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:33.486991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:33.497699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Jul 19 12:21:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 19 12:21:33.503516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:33.563667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:33.601259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea4811-67f7-484e-b70c-b6e0917f48ce None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:33.634946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:33.635726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:33.636007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:33.636310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:33.646562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:33.647994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 19 12:21:33.664803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:33.665844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:33.666967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 19 12:21:33.666967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 19 12:21:33.666967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 19 12:21:33.666967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:33.666967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 650/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:33.671777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:33.772806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610d50d-bca4-46ab-bc11-fea8547296af None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:33.805209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:33.805876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:33.806152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:33.806435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:33.807151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:33.807462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:33.808858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:33.820354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Jul 19 12:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:33.826123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:33.923296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b047128-3d44-4e7e-8388-6e13fce8bc80 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:33.936839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:33.957595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:33.958366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:33.958647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:33.958919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:33.965342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:33.966775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 19 12:21:33.980101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:33.981148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: ERROR keystone.server.flask.application Jul 19 12:21:33.982204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 651/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Jul 19 12:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:33.986945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:34.035987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:34.085412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f81e94-1997-4b02-aa4f-9b23bc0b5d8c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:34.117871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:34.118517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:34.118799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:34.119074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:34.119847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:34.120254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:34.121674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de2758-0cb2-40e4-85a9-8a46aa4ef629 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:34.133697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Jul 19 12:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:34.139505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:34.235257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0605f-1981-41ad-9967-5463323960a6 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:34.268379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65add6df-2c31-4991-905b-90d53514817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:34.269042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add6df-2c31-4991-905b-90d53514817a None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:34.269317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add6df-2c31-4991-905b-90d53514817a None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:34.269587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65add6df-2c31-4991-905b-90d53514817a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:34.270401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add6df-2c31-4991-905b-90d53514817a None admin] RBAC: Adding query filter params (role.id=ac904f211a8f450fbf556c879296af4b, scope.project.id=b1a666f3a1a144858853d303838a22b9, user.id=06d3500f0bda4ec58b2d561f13eb42d9) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:34.270708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add6df-2c31-4991-905b-90d53514817a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:34.272127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65add6df-2c31-4991-905b-90d53514817a None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:34.277812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 652/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Jul 19 12:21:34 2022] GET /identity/v3/role_assignments?user.id=06d3500f0bda4ec58b2d561f13eb42d9&scope.project.id=b1a666f3a1a144858853d303838a22b9&role.id=ac904f211a8f450fbf556c879296af4b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:36.342127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408a0b8-3e88-4281-8032-811afedbd4da None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:36.342759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408a0b8-3e88-4281-8032-811afedbd4da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:36.342759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408a0b8-3e88-4281-8032-811afedbd4da None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:36.344140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1288] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:36.352824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:36.353473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:36.353805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:36.374030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:36.375340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:36.411302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:36.453720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 653/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 19 12:21:36.476871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408a0b8-3e88-4281-8032-811afedbd4da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:36.574469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408a0b8-3e88-4281-8032-811afedbd4da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:36.585235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:36.608700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:36.609429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:36.609716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:36.609994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:36.611089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:36.611995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:36.624048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:36.625306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:36.626473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:21:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:36.631733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:36.726528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155673fe-af21-4f60-8fc5-e748b05b86fb None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:36.759423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:36.760088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:36.760399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:36.760677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:36.761382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:36.761687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:36.763066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5133e-b574-4ef0-aa81-1119129a0eeb None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:36.767982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 654/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jul 19 12:21:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:36.774556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:36.878955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defd14f-41e2-42b6-a686-eaaa4147c9a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:36.911429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:36.912083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:36.912398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:36.912680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:36.913380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] RBAC: Adding query filter params (type=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:36.913692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:36.915056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bfcfb-2a5f-423d-b279-dce4df269fab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:36.919437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jul 19 12:21:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:38.895069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108108-f799-4fce-ae46-68d7bc193016 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:38.895570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108108-f799-4fce-ae46-68d7bc193016 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:38.895570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64108108-f799-4fce-ae46-68d7bc193016 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:38.896933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 655/1293] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:38.902073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99be45-daeb-4216-85d6-ee4af532fdbb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:38.902449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99be45-daeb-4216-85d6-ee4af532fdbb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:38.902632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99be45-daeb-4216-85d6-ee4af532fdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:38.919269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6e99be45-daeb-4216-85d6-ee4af532fdbb None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:38.920465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e99be45-daeb-4216-85d6-ee4af532fdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:38.950299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:38.986166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:39.008892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108108-f799-4fce-ae46-68d7bc193016 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:39.039450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:39.106766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64108108-f799-4fce-ae46-68d7bc193016 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:39.139443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fbdf7-0449-4980-ad4d-e8e4f482c533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:39.140096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fbdf7-0449-4980-ad4d-e8e4f482c533 None admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:39.140406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fbdf7-0449-4980-ad4d-e8e4f482c533 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:39.140810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fbdf7-0449-4980-ad4d-e8e4f482c533 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:39.141408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fbdf7-0449-4980-ad4d-e8e4f482c533 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:39.142770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fbdf7-0449-4980-ad4d-e8e4f482c533 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:39.156209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 656/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Jul 19 12:21:39 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:21:41.194149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cc80-daa0-4de6-9651-9e8c8dce2e81 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:41.194695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cc80-daa0-4de6-9651-9e8c8dce2e81 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:41.194695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cc80-daa0-4de6-9651-9e8c8dce2e81 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:41.196059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1296] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:41.201242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:41.201651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:41.201800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:41.218529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:41.219689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:41.288650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 657/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:41.311087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cc80-daa0-4de6-9651-9e8c8dce2e81 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:41.424031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cc80-daa0-4de6-9651-9e8c8dce2e81 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:41.463405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:41.464128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:41.464449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:41.464733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:41.465419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:41.466796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:41.478586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:41.479743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:41.480960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:21:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:41.485835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:41.505110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:41.583461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02b067-8c28-4124-a20c-f845e9937d1e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:41.618526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:41.619184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:41.619466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:41.619737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:41.620463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:41.620765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:41.622128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:41.627110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 658/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jul 19 12:21:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:41.633709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:41.664198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:41.731010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b35eae-eb4c-464c-9ce2-7b9d3f3b7b8f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:41.764324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:41.764959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:41.765244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:41.765524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:41.766301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] RBAC: Adding query filter params (interface=public, service_id=d161777ce99443e8861fe4a087914f88, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:41.766617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:41.767974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6acef8-c19e-4792-ab78-8111aa50a710 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:41.772937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Jul 19 12:21:41 2022] GET /identity/v3/endpoints?service_id=d161777ce99443e8861fe4a087914f88&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:41.778370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:41.857811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:41.876253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d42a00-09dc-4489-8b01-283615e6a504 None admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:41.909762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d2ac6-b6fe-4d7d-a34a-97ef56ae8030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:41.910412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d2ac6-b6fe-4d7d-a34a-97ef56ae8030 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:41.910697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d2ac6-b6fe-4d7d-a34a-97ef56ae8030 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:41.910973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d2ac6-b6fe-4d7d-a34a-97ef56ae8030 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:41.911646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d2ac6-b6fe-4d7d-a34a-97ef56ae8030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:41.913056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d2ac6-b6fe-4d7d-a34a-97ef56ae8030 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:41.919663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 659/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Jul 19 12:21:41 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:21:43.930555 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51d93-c300-428b-b58f-bb45ab8144ca None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:43.931063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51d93-c300-428b-b58f-bb45ab8144ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:43.931063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b51d93-c300-428b-b58f-bb45ab8144ca None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:43.932434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1302] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:21:43.937567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:43.937980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:43.938122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:43.954331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:21:43.955492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:21:44.019265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 660/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:21:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 19 12:21:44.042353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b51d93-c300-428b-b58f-bb45ab8144ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:44.138633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:44.146246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b51d93-c300-428b-b58f-bb45ab8144ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:44.178659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:44.179443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:44.179729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:44.180005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:44.180737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:44.182045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 19 12:21:44.194185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 19 12:21:44.195353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 19 12:21:44.196464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Jul 19 12:21:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 19 12:21:44.201324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:44.275028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:44.296951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6efa1e-6bb5-48c2-bbfc-7c3001128352 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:44.329322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:44.329966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:44.330254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:44.330530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] PATH_INFO: `/v3/services` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:44.331277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:21:44.331579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:44.332967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963435c-939a-46f3-b2f9-7f8ac3479514 None admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:44.338254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 661/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Jul 19 12:21:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:21:44.344250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:21:44.444820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78c7d7-5af5-4de3-8f4c-edabb2fc7953 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:21:44.476551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b843ef-3f51-4422-8c2f-baba0ee61370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:21:44.477190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b843ef-3f51-4422-8c2f-baba0ee61370 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:21:44.477472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b843ef-3f51-4422-8c2f-baba0ee61370 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:21:44.477749 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b843ef-3f51-4422-8c2f-baba0ee61370 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:21:44.478476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b843ef-3f51-4422-8c2f-baba0ee61370 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:21:44.479833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b843ef-3f51-4422-8c2f-baba0ee61370 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:21:44.497386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Jul 19 12:21:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:21:54.334995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=30 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:21:54.493195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:46.599739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9401-9af3-4f36-9dc3-d554759f5923 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:46.600293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9401-9af3-4f36-9dc3-d554759f5923 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:46.600895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c9401-9af3-4f36-9dc3-d554759f5923 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:46.602274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 662/1307] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:22:46.611095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559829e-ac73-49c5-9df5-a3fd020da0ce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:46.611095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559829e-ac73-49c5-9df5-a3fd020da0ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:46.611095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559829e-ac73-49c5-9df5-a3fd020da0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:46.715333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8559829e-ac73-49c5-9df5-a3fd020da0ce None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:22:46.716587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8559829e-ac73-49c5-9df5-a3fd020da0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:46.846649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:22:46.898548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0cd58-270e-4681-aac2-4071ca2f3ee8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:46.898843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0cd58-270e-4681-aac2-4071ca2f3ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:46.899349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0cd58-270e-4681-aac2-4071ca2f3ee8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:46.900749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 663/1309] 10.222.0.13 () {54 vars in 965 bytes} [Tue Jul 19 12:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:22:46.907711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e809c-5c62-4828-b1a3-462a0ca8f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:46.908085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e809c-5c62-4828-b1a3-462a0ca8f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:46.908470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e809c-5c62-4828-b1a3-462a0ca8f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:46.999868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca1e809c-5c62-4828-b1a3-462a0ca8f7d2 None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:22:47.001971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca1e809c-5c62-4828-b1a3-462a0ca8f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:47.023590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:47.083014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1310] 10.222.0.13 () {58 vars in 1062 bytes} [Tue Jul 19 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:22:47.092490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0cd58-270e-4681-aac2-4071ca2f3ee8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:22:47.094306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-03d0cd58-270e-4681-aac2-4071ca2f3ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:47.207005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0cd58-270e-4681-aac2-4071ca2f3ee8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:22:47.288495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:22:47.289830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:47.290049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:47.290594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:47.292442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:47.357786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:47.425481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:22:47.427319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e9f57-07c3-42d9-9b65-31c03bb6f5e3 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:22:47.454549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 664/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:22:49.860874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cbc8f-f1c4-42a0-afde-41d3de14c581 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:49.861482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cbc8f-f1c4-42a0-afde-41d3de14c581 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:49.861482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cbc8f-f1c4-42a0-afde-41d3de14c581 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:49.862758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1312] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:22:49.868581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc73ec2-ea59-4071-88b3-c6812adff48c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:49.868992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc73ec2-ea59-4071-88b3-c6812adff48c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:49.869139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc73ec2-ea59-4071-88b3-c6812adff48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:49.943443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5cc73ec2-ea59-4071-88b3-c6812adff48c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:22:49.944723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5cc73ec2-ea59-4071-88b3-c6812adff48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:50.041907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 665/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:22:50.080598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cbc8f-f1c4-42a0-afde-41d3de14c581 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:22:50.082134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-103cbc8f-f1c4-42a0-afde-41d3de14c581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:50.190909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cbc8f-f1c4-42a0-afde-41d3de14c581 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:22:50.279322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:50.282911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:22:50.283632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:50.284362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:50.285058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:50.287316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:50.407390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:22:50.408883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45988367-0db8-4399-ad91-1e914bc88323 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:22:50.430794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:22:52.796293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09126d-ccd4-4b11-8669-23d794dc5d54 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:52.797403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09126d-ccd4-4b11-8669-23d794dc5d54 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:52.797403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09126d-ccd4-4b11-8669-23d794dc5d54 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:52.798732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 666/1315] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:22:52.803782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba6cb0-fff0-4cfc-bced-4fa7b2aec1ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:52.804148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba6cb0-fff0-4cfc-bced-4fa7b2aec1ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:52.804396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba6cb0-fff0-4cfc-bced-4fa7b2aec1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:52.868311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8aba6cb0-fff0-4cfc-bced-4fa7b2aec1ff None None] MFA Rules not processed for user `8cc5b62ae82b42d4bdf8ad4af8b91bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:22:52.869453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8aba6cb0-fff0-4cfc-bced-4fa7b2aec1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:52.923835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:52.967886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:22:53.008417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09126d-ccd4-4b11-8669-23d794dc5d54 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:22:53.009927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cb09126d-ccd4-4b11-8669-23d794dc5d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:53.020038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:53.111238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09126d-ccd4-4b11-8669-23d794dc5d54 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:22:53.181334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:22:53.181912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:53.182193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:53.182470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:53.184063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:53.246967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:53.292572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:22:53.294009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16562a-6b71-4bad-95a3-0f7e83210b10 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:22:53.312104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 667/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:22:55.886041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0b90-3596-4b30-bd71-310481941628 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:55.886565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0b90-3596-4b30-bd71-310481941628 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:55.886565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0b90-3596-4b30-bd71-310481941628 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:55.887900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1318] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:22:55.893700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c388c95-4c3b-419d-9088-d2aab5be6dcc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:55.894125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c388c95-4c3b-419d-9088-d2aab5be6dcc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:55.894269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c388c95-4c3b-419d-9088-d2aab5be6dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:55.957841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4c388c95-4c3b-419d-9088-d2aab5be6dcc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:22:55.959047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c388c95-4c3b-419d-9088-d2aab5be6dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:56.053130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 668/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:22:56.092012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0b90-3596-4b30-bd71-310481941628 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:22:56.094199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a79b0b90-3596-4b30-bd71-310481941628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:56.194641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0b90-3596-4b30-bd71-310481941628 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:22:56.266423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:22:56.267078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:56.267365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:56.267648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:56.275870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:56.283589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:56.387971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:22:56.389448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88383b6-3572-4910-9c6d-eb7364359af7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:22:56.407708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:22:59.126047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bb8fa-62d7-48f2-8022-841156a8e181 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:59.126617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bb8fa-62d7-48f2-8022-841156a8e181 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:59.126736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bb8fa-62d7-48f2-8022-841156a8e181 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:59.128058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 669/1321] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:22:59.133455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0723d6-b900-40cc-8c3b-539345b984e8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:59.133888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0723d6-b900-40cc-8c3b-539345b984e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:59.134029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0723d6-b900-40cc-8c3b-539345b984e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:59.197920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0a0723d6-b900-40cc-8c3b-539345b984e8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:22:59.199170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a0723d6-b900-40cc-8c3b-539345b984e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:59.263655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:59.309506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:22:59.347314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bb8fa-62d7-48f2-8022-841156a8e181 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:22:59.349077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a53bb8fa-62d7-48f2-8022-841156a8e181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:59.370178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:59.455436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bb8fa-62d7-48f2-8022-841156a8e181 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:22:59.529091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:22:59.529739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:22:59.530026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:22:59.530305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:22:59.532606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:22:59.607371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:22:59.650002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:22:59.651754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7afc43-7faa-468c-a0e8-edb0b0b1d775 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:22:59.672292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 670/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:01.772805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66209882-63b9-4d35-883f-9c2f901f5fab None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:01.773358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66209882-63b9-4d35-883f-9c2f901f5fab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:01.773358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66209882-63b9-4d35-883f-9c2f901f5fab None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:01.774747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1324] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:01.780132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3212-6179-42e2-8918-b8cc48e2a44e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:01.780580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3212-6179-42e2-8918-b8cc48e2a44e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:01.780933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3212-6179-42e2-8918-b8cc48e2a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:01.846513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-68ec3212-6179-42e2-8918-b8cc48e2a44e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:01.847785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-68ec3212-6179-42e2-8918-b8cc48e2a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:01.944301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 671/1325] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:01.980243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66209882-63b9-4d35-883f-9c2f901f5fab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:01.981804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66209882-63b9-4d35-883f-9c2f901f5fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:02.086799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66209882-63b9-4d35-883f-9c2f901f5fab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:02.163345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:02.163999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:02.164318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:02.164599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:02.166227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:02.183238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:02.291069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:02.292673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdd07d-56ef-4f94-8f12-975f21fad23d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:02.310615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1326] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:04.410897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2af30-7d8e-41c3-978e-71de33ca9809 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:04.411391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2af30-7d8e-41c3-978e-71de33ca9809 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:04.411514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2af30-7d8e-41c3-978e-71de33ca9809 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:04.412914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 672/1327] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:04.418365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3a665-141a-4713-92ba-83d3217da7e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:04.418810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3a665-141a-4713-92ba-83d3217da7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:04.418925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3a665-141a-4713-92ba-83d3217da7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:04.487918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39c3a665-141a-4713-92ba-83d3217da7e2 None None] MFA Rules not processed for user `8cc5b62ae82b42d4bdf8ad4af8b91bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:04.489198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39c3a665-141a-4713-92ba-83d3217da7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:04.554484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:04.589271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1328] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:04.631946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2af30-7d8e-41c3-978e-71de33ca9809 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:04.633496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f3a2af30-7d8e-41c3-978e-71de33ca9809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:04.664550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:04.739434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2af30-7d8e-41c3-978e-71de33ca9809 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:04.812041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:04.812721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:04.813003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:04.813282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:04.814913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:04.910637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:04.953318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:04.954781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59271c-29c0-4a78-98c7-c4d9fcb60470 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:04.973939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 673/1329] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:07.279013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175c23-11f3-4c38-ba44-e9dc72300083 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:07.279571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175c23-11f3-4c38-ba44-e9dc72300083 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:07.279571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175c23-11f3-4c38-ba44-e9dc72300083 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:07.281035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1330] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:07.286478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea18d7b-14fa-4697-a26a-61863f32f032 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:07.286886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea18d7b-14fa-4697-a26a-61863f32f032 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:07.287049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea18d7b-14fa-4697-a26a-61863f32f032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:07.358386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dea18d7b-14fa-4697-a26a-61863f32f032 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:07.360164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dea18d7b-14fa-4697-a26a-61863f32f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:07.456500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 674/1331] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:07.493176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175c23-11f3-4c38-ba44-e9dc72300083 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:07.494735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8a175c23-11f3-4c38-ba44-e9dc72300083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:07.596765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175c23-11f3-4c38-ba44-e9dc72300083 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:07.676154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:07.676829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:07.677116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:07.677396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:07.679040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:07.704985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:07.786496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:07.787865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3edf91-a053-497e-a910-4ffef04890ad service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:07.805627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1332] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:09.898379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95e72b-1e17-4732-8455-8f98eae7cc92 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:09.898904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95e72b-1e17-4732-8455-8f98eae7cc92 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:09.898904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95e72b-1e17-4732-8455-8f98eae7cc92 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:09.900307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 675/1333] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:09.905427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5031e-bd89-4cb1-949c-806027b05687 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:09.905837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5031e-bd89-4cb1-949c-806027b05687 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:09.905996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b5031e-bd89-4cb1-949c-806027b05687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:09.968827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-00b5031e-bd89-4cb1-949c-806027b05687 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:09.970018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-00b5031e-bd89-4cb1-949c-806027b05687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:10.042583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:10.065607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1334] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:10.105241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95e72b-1e17-4732-8455-8f98eae7cc92 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:10.106827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db95e72b-1e17-4732-8455-8f98eae7cc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:10.147656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:10.210763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95e72b-1e17-4732-8455-8f98eae7cc92 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:10.281961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:10.282601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:10.282882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:10.283161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:10.284795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:10.366431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:10.391725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:10.393205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bc07e-98ca-4a01-8573-fc2dfebf413b service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:10.411246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 676/1335] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:12.787868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38a527-9ed8-4fe6-823e-61220fe90ad9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:12.788454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38a527-9ed8-4fe6-823e-61220fe90ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:12.788454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38a527-9ed8-4fe6-823e-61220fe90ad9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:12.789812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1336] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:12.794897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461be35-2f40-4087-b86a-c2e4d4efe6da None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:12.795316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461be35-2f40-4087-b86a-c2e4d4efe6da None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:12.795459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461be35-2f40-4087-b86a-c2e4d4efe6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:12.858515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8461be35-2f40-4087-b86a-c2e4d4efe6da None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:12.859681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8461be35-2f40-4087-b86a-c2e4d4efe6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:12.954606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 677/1337] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:12.990297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38a527-9ed8-4fe6-823e-61220fe90ad9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:12.991797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b38a527-9ed8-4fe6-823e-61220fe90ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:13.092611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38a527-9ed8-4fe6-823e-61220fe90ad9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:13.164345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:13.164987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:13.165275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:13.165553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:13.167170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:13.203401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:13.276493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:13.277937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d8e09-4d9f-4263-aab0-9fcf7b0b7a7a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:13.295438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1338] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:17.261089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a94b-4fdc-48a4-93af-cb81e3d945d3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:17.261589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a94b-4fdc-48a4-93af-cb81e3d945d3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:17.261667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05a94b-4fdc-48a4-93af-cb81e3d945d3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:17.263034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 678/1339] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:17.268977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b13bcf-5312-4c35-8d76-564db378a409 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:17.269407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b13bcf-5312-4c35-8d76-564db378a409 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:17.269550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b13bcf-5312-4c35-8d76-564db378a409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:17.334396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-21b13bcf-5312-4c35-8d76-564db378a409 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:17.335517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21b13bcf-5312-4c35-8d76-564db378a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:17.426107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:17.438925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1340] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:17.478417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a94b-4fdc-48a4-93af-cb81e3d945d3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:17.479930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c05a94b-4fdc-48a4-93af-cb81e3d945d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:17.528100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:17.584775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05a94b-4fdc-48a4-93af-cb81e3d945d3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:17.661928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:17.662577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:17.662857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:17.663129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:17.664766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:17.757732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:17.774157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:17.775598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106d58b-4f40-4c6b-b847-1ca492f8cdce service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:17.794711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 679/1341] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:20.484775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58306891-337c-4a61-b8a1-ac192e6b3f92 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:20.485322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58306891-337c-4a61-b8a1-ac192e6b3f92 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:20.485322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58306891-337c-4a61-b8a1-ac192e6b3f92 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:20.486754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1342] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:20.492696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9f62-f8d5-4c97-8334-45e6e6b6a6d6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:20.493211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9f62-f8d5-4c97-8334-45e6e6b6a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:20.493307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9f62-f8d5-4c97-8334-45e6e6b6a6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:20.562714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8f3f9f62-f8d5-4c97-8334-45e6e6b6a6d6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:20.563904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8f3f9f62-f8d5-4c97-8334-45e6e6b6a6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:20.664935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 680/1343] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:20.704211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58306891-337c-4a61-b8a1-ac192e6b3f92 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:20.705732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58306891-337c-4a61-b8a1-ac192e6b3f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:20.807743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58306891-337c-4a61-b8a1-ac192e6b3f92 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:20.881881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:20.882534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:20.882824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:20.883106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:20.884781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:20.928007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:21.003684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:21.005227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e22473-3e12-496f-8f21-d205af12694b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:21.026296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1344] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:25.027608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48151090-01ec-4e29-b670-d5f3f82eceb8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:25.028078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48151090-01ec-4e29-b670-d5f3f82eceb8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:25.028194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48151090-01ec-4e29-b670-d5f3f82eceb8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:25.029567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 681/1345] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:25.035040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee738-d005-47fa-b1b9-1ffa22922eda None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:25.035462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee738-d005-47fa-b1b9-1ffa22922eda None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:25.035612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee738-d005-47fa-b1b9-1ffa22922eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:25.104612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b1eee738-d005-47fa-b1b9-1ffa22922eda None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:25.105475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b1eee738-d005-47fa-b1b9-1ffa22922eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:25.223033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:25.226045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1346] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:25.266099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48151090-01ec-4e29-b670-d5f3f82eceb8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:25.267185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-48151090-01ec-4e29-b670-d5f3f82eceb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:25.335335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:25.397642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48151090-01ec-4e29-b670-d5f3f82eceb8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:25.485938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:25.486596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:25.486883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:25.487626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:25.488905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:25.631908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:25.636845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:25.638310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6503fc6-3e2f-4779-9a06-759402db19b5 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:25.658890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 682/1347] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:30.504741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0887-8b80-4d27-a4ce-bba8198537ed None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:30.505248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0887-8b80-4d27-a4ce-bba8198537ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:30.505549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0887-8b80-4d27-a4ce-bba8198537ed None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:30.507187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1348] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:30.512271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7a529-4149-4751-ba0a-369f27803cb7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:30.513003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7a529-4149-4751-ba0a-369f27803cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:30.513423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7a529-4149-4751-ba0a-369f27803cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:30.601736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-80c7a529-4149-4751-ba0a-369f27803cb7 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:30.603207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-80c7a529-4149-4751-ba0a-369f27803cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:30.775825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 683/1349] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:23:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:34.884698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ae392-5f22-43f7-886f-74748fe5dd1e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:34.885678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ae392-5f22-43f7-886f-74748fe5dd1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:34.886192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ae392-5f22-43f7-886f-74748fe5dd1e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:34.888078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1350] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:34.893554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ef3f-83df-4b5e-81b8-215434a44947 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:34.894358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ef3f-83df-4b5e-81b8-215434a44947 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:34.894721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73ef3f-83df-4b5e-81b8-215434a44947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:34.952645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:34.979159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1f73ef3f-83df-4b5e-81b8-215434a44947 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:34.980564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1f73ef3f-83df-4b5e-81b8-215434a44947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:35.092781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 684/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:36.496288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567babe6-0f58-4183-92de-e15a76d9ed5e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:36.496831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567babe6-0f58-4183-92de-e15a76d9ed5e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:36.496831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567babe6-0f58-4183-92de-e15a76d9ed5e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:36.498311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1352] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:36.503218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf486a-b96f-4b9f-a771-79875bbd2cae None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:36.503621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf486a-b96f-4b9f-a771-79875bbd2cae None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:36.503907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf486a-b96f-4b9f-a771-79875bbd2cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:36.578106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-42bf486a-b96f-4b9f-a771-79875bbd2cae None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:36.579289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-42bf486a-b96f-4b9f-a771-79875bbd2cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:36.628650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:36.694550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 685/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:23:40.259591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13dfa-77d8-4565-9939-d79f1308621b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:40.260164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13dfa-77d8-4565-9939-d79f1308621b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:40.260164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb13dfa-77d8-4565-9939-d79f1308621b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:40.261706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1354] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:40.267661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8b05e-f153-4dc6-a6f3-691709707495 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:40.268103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8b05e-f153-4dc6-a6f3-691709707495 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:40.268261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8b05e-f153-4dc6-a6f3-691709707495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:40.336901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-45a8b05e-f153-4dc6-a6f3-691709707495 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:40.338330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-45a8b05e-f153-4dc6-a6f3-691709707495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:40.446555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 686/1355] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:40.500957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688d3e0-f568-4a20-9e85-096ba4e89c89 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:40.501558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688d3e0-f568-4a20-9e85-096ba4e89c89 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:40.502049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688d3e0-f568-4a20-9e85-096ba4e89c89 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:40.504042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1356] 10.222.0.13 () {54 vars in 962 bytes} [Tue Jul 19 12:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:40.514201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2250128-86e7-40d5-99d2-ed82a30f38fb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:40.515113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2250128-86e7-40d5-99d2-ed82a30f38fb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:40.517484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2250128-86e7-40d5-99d2-ed82a30f38fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:40.536022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:40.602058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b2250128-86e7-40d5-99d2-ed82a30f38fb None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:40.603177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b2250128-86e7-40d5-99d2-ed82a30f38fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:40.737560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 687/1357] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Jul 19 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:23:40.745817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688d3e0-f568-4a20-9e85-096ba4e89c89 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:40.748206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a688d3e0-f568-4a20-9e85-096ba4e89c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:40.904892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688d3e0-f568-4a20-9e85-096ba4e89c89 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:40.981992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:41.026762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:41.027560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:41.027949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:41.028361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:41.031473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:41.126082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:41.127856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5c9ee-b033-438e-a468-f9b318111d1e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:41.147149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1358] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:23:43.401040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0a462-b151-44b3-9830-60113008dc7e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:43.401587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0a462-b151-44b3-9830-60113008dc7e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:43.401587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0a462-b151-44b3-9830-60113008dc7e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:43.403024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 688/1359] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:43.408961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410901a-3feb-487c-96ba-9bb3a3fa0a2d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:43.409363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410901a-3feb-487c-96ba-9bb3a3fa0a2d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:43.409552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410901a-3feb-487c-96ba-9bb3a3fa0a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:43.480844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1410901a-3feb-487c-96ba-9bb3a3fa0a2d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:43.482075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1410901a-3feb-487c-96ba-9bb3a3fa0a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:43.524919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:43.596480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1360] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:43.646065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0a462-b151-44b3-9830-60113008dc7e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:43.649771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20a0a462-b151-44b3-9830-60113008dc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:43.753124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:43.861890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0a462-b151-44b3-9830-60113008dc7e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:43.983313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:43.983962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:43.984269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:43.984547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:43.986319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:44.010794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:44.081059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:44.082733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c01bc0d-6a8c-4d1f-839e-2a34a6c0a0f6 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:44.102286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 689/1361] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:46.548657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5b105-c65f-4a90-be4e-4c02095c059b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:46.549225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5b105-c65f-4a90-be4e-4c02095c059b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:46.549225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5b105-c65f-4a90-be4e-4c02095c059b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:46.550750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1362] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:46.555947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33785a-e961-41ac-9b08-7e1a01869569 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:46.556385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33785a-e961-41ac-9b08-7e1a01869569 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:46.556598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33785a-e961-41ac-9b08-7e1a01869569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:46.625450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0f33785a-e961-41ac-9b08-7e1a01869569 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:46.626673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0f33785a-e961-41ac-9b08-7e1a01869569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:46.725287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:46.739521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 690/1363] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:46.777833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f6636-d033-4dc8-af4e-47b6140d9d01 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:46.778121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f6636-d033-4dc8-af4e-47b6140d9d01 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:46.778610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f6636-d033-4dc8-af4e-47b6140d9d01 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:46.780007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1364] 10.222.0.13 () {54 vars in 962 bytes} [Tue Jul 19 12:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:46.785456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aeb6a-d18e-458f-a2f9-e13bcb7b5cfc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:46.785861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aeb6a-d18e-458f-a2f9-e13bcb7b5cfc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:46.786249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aeb6a-d18e-458f-a2f9-e13bcb7b5cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:46.882723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b01aeb6a-d18e-458f-a2f9-e13bcb7b5cfc None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:46.883925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b01aeb6a-d18e-458f-a2f9-e13bcb7b5cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:47.009802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 691/1365] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Jul 19 12:23:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:47.019664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f6636-d033-4dc8-af4e-47b6140d9d01 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:47.021254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-558f6636-d033-4dc8-af4e-47b6140d9d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:47.173430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f6636-d033-4dc8-af4e-47b6140d9d01 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:47.194905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:47.291661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:47.292337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:47.292634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:47.292908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:47.294558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:47.392155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:47.393800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddff330-c859-4886-a1c9-08814ef11445 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:47.413093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1366] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:48.274362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc289c8-d66e-4b74-8a5e-220f298f465c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:48.274914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc289c8-d66e-4b74-8a5e-220f298f465c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:48.275203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc289c8-d66e-4b74-8a5e-220f298f465c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:48.277027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 692/1367] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:48.282629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6561a-4e07-41d0-a126-2a6b0a282cfc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:48.283089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6561a-4e07-41d0-a126-2a6b0a282cfc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:48.283387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6561a-4e07-41d0-a126-2a6b0a282cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:48.316908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:48.357802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-11c6561a-4e07-41d0-a126-2a6b0a282cfc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:48.359144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11c6561a-4e07-41d0-a126-2a6b0a282cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:48.460223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1368] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:48.801783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8eae-98e7-46ae-b6ea-a71de109cbb4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:48.801783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8eae-98e7-46ae-b6ea-a71de109cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:48.801783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380f8eae-98e7-46ae-b6ea-a71de109cbb4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:48.804409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 693/1369] 10.222.0.13 () {54 vars in 964 bytes} [Tue Jul 19 12:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:48.807694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dab7b-dda0-4d5a-941f-a5874f636f33 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:48.807994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dab7b-dda0-4d5a-941f-a5874f636f33 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:48.808325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dab7b-dda0-4d5a-941f-a5874f636f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:48.889538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-463dab7b-dda0-4d5a-941f-a5874f636f33 None None] MFA Rules not processed for user `a58d153dd25943ef80eea95893118e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:48.890776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-463dab7b-dda0-4d5a-941f-a5874f636f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:48.939968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:48.972564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1370] 10.222.0.13 () {58 vars in 1061 bytes} [Tue Jul 19 12:23:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:48.980169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f8eae-98e7-46ae-b6ea-a71de109cbb4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:48.981741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-380f8eae-98e7-46ae-b6ea-a71de109cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:49.035069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:49.094749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380f8eae-98e7-46ae-b6ea-a71de109cbb4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:49.178256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:49.178948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:49.179261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:49.179528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:49.182250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:49.286055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:49.299835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:49.301344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b08ba-4fe6-4d04-a4ee-2ffca15438f1 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:49.320047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 694/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:49.531785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4ed2-e128-4b4c-b471-b89f4eb5dff7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:49.532246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4ed2-e128-4b4c-b471-b89f4eb5dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:49.532790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4ed2-e128-4b4c-b471-b89f4eb5dff7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:49.535203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1372] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:49.539933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541e744-9036-495d-abbb-713ac9e97434 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:49.540146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541e744-9036-495d-abbb-713ac9e97434 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:49.540457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f541e744-9036-495d-abbb-713ac9e97434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:49.617905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f541e744-9036-495d-abbb-713ac9e97434 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:49.619152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f541e744-9036-495d-abbb-713ac9e97434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:49.723011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 695/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:49.767934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af4ed2-e128-4b4c-b471-b89f4eb5dff7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:49.769474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67af4ed2-e128-4b4c-b471-b89f4eb5dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:49.914504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af4ed2-e128-4b4c-b471-b89f4eb5dff7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:49.967088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:50.031377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:50.032037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:50.032355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:50.032636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:50.034364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:50.127065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:50.128697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5112cf45-3cc4-4af4-9ac4-052ad5ca3cad service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:50.146609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1374] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:52.275861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f376946-344f-4fd3-8d08-4fe14267c215 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:52.276437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f376946-344f-4fd3-8d08-4fe14267c215 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:52.276437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f376946-344f-4fd3-8d08-4fe14267c215 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:52.277819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 696/1375] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:52.283901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636cc5c-e704-44ba-9eff-aee3aa70f8af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:52.284346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636cc5c-e704-44ba-9eff-aee3aa70f8af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:52.284507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636cc5c-e704-44ba-9eff-aee3aa70f8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:52.361938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a636cc5c-e704-44ba-9eff-aee3aa70f8af None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:52.363178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a636cc5c-e704-44ba-9eff-aee3aa70f8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:52.380066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:52.475890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1376] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:52.520862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f376946-344f-4fd3-8d08-4fe14267c215 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:52.522361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f376946-344f-4fd3-8d08-4fe14267c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:52.582438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:52.678688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f376946-344f-4fd3-8d08-4fe14267c215 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:52.822582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:52.822582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:52.823187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:52.823566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:52.826042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:52.855072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:52.941765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:52.943619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736c9d-076b-4dc7-8a0a-b606157e57b2 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:52.969380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 697/1377] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:55.054812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74975d5-0c55-4e83-88b9-23e70706d89d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:55.055365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74975d5-0c55-4e83-88b9-23e70706d89d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:55.055365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74975d5-0c55-4e83-88b9-23e70706d89d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:55.056920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1378] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:55.062032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1092d-ce92-4a4c-a25e-c608ed45a6ea None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:55.062431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1092d-ce92-4a4c-a25e-c608ed45a6ea None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:55.062599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1092d-ce92-4a4c-a25e-c608ed45a6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:55.129302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2aa1092d-ce92-4a4c-a25e-c608ed45a6ea None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:55.130519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2aa1092d-ce92-4a4c-a25e-c608ed45a6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:55.233586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:55.240755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 698/1379] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:55.285030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74975d5-0c55-4e83-88b9-23e70706d89d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:55.286827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a74975d5-0c55-4e83-88b9-23e70706d89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:55.438145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:55.449637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74975d5-0c55-4e83-88b9-23e70706d89d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:55.568297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:55.568956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:55.569249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:55.569532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:55.571179 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:55.668569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:55.670211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6215ea-c07c-4457-807f-07f1fd204367 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:55.681351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:55.691295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1380] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:23:57.770140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147ffe6-db12-4aa8-aed6-da3f672f91f3 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:57.770704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147ffe6-db12-4aa8-aed6-da3f672f91f3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:57.770704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147ffe6-db12-4aa8-aed6-da3f672f91f3 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:57.772054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 699/1381] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:23:57.778057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71953-b0fe-41c0-b477-15a02bcaa125 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:57.778462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71953-b0fe-41c0-b477-15a02bcaa125 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:57.778798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71953-b0fe-41c0-b477-15a02bcaa125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:57.863248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2e71953-b0fe-41c0-b477-15a02bcaa125 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:23:57.864710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2e71953-b0fe-41c0-b477-15a02bcaa125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:57.971760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1382] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:23:58.014181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147ffe6-db12-4aa8-aed6-da3f672f91f3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:23:58.016235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8147ffe6-db12-4aa8-aed6-da3f672f91f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:58.184731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147ffe6-db12-4aa8-aed6-da3f672f91f3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:23:58.267787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:23:58.323316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:23:58.323979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:23:58.324313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:23:58.324585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:23:58.326253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:23:58.420476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:23:58.422112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df79347-102e-4aa4-b531-7ad51000b722 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:23:58.440570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 700/1383] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:24:00.500647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f2ce2-cb47-49b7-a191-85fe1f122518 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:00.501326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f2ce2-cb47-49b7-a191-85fe1f122518 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:00.501326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f2ce2-cb47-49b7-a191-85fe1f122518 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:00.502610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1384] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:24:00.509050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461dda9-e493-42a7-9ee6-b545fff0ae1e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:00.509455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461dda9-e493-42a7-9ee6-b545fff0ae1e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:00.509623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461dda9-e493-42a7-9ee6-b545fff0ae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:00.576765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b461dda9-e493-42a7-9ee6-b545fff0ae1e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:00.577983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b461dda9-e493-42a7-9ee6-b545fff0ae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:00.629260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:00.710019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 701/1385] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:00.755533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f2ce2-cb47-49b7-a191-85fe1f122518 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:00.757125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-557f2ce2-cb47-49b7-a191-85fe1f122518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:00.836255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:00.910226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f2ce2-cb47-49b7-a191-85fe1f122518 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:01.032199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:01.032773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:01.033136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:01.033419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:01.035193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:01.076291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:01.131841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:01.133486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c5534-fd2c-4f6b-9c57-ae4bf955faa0 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:01.153815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1386] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:24:03.589644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4893d-3e80-4d3c-b8de-312a1060dd91 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:03.590265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4893d-3e80-4d3c-b8de-312a1060dd91 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:03.590265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4893d-3e80-4d3c-b8de-312a1060dd91 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:03.591810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 702/1387] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:03.597838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49131091-c497-47a8-bfa7-aae084589cc2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:03.598246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49131091-c497-47a8-bfa7-aae084589cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:03.598491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49131091-c497-47a8-bfa7-aae084589cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:03.674662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49131091-c497-47a8-bfa7-aae084589cc2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:03.676489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49131091-c497-47a8-bfa7-aae084589cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:03.776924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1388] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:03.821947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4893d-3e80-4d3c-b8de-312a1060dd91 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:03.823530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f7f4893d-3e80-4d3c-b8de-312a1060dd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:03.968391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4893d-3e80-4d3c-b8de-312a1060dd91 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:03.985146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:04.084665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:04.085592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:04.085878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:04.086151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:04.087765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:04.175782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:04.177417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274c41e-8528-4ff5-8e5f-d2033e4bd194 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:04.195530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 703/1389] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:04.936545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1273209-522c-4080-91e4-de1454b2d4b6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:04.937068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1273209-522c-4080-91e4-de1454b2d4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:04.937068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1273209-522c-4080-91e4-de1454b2d4b6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:04.938553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1390] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:04.943567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10097f26-eb50-4d9e-ac3f-3f0c7013b9a0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:04.943995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10097f26-eb50-4d9e-ac3f-3f0c7013b9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:04.944107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10097f26-eb50-4d9e-ac3f-3f0c7013b9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:04.969091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:05.011856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-10097f26-eb50-4d9e-ac3f-3f0c7013b9a0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:05.013096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-10097f26-eb50-4d9e-ac3f-3f0c7013b9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:05.108453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 704/1391] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:05.164255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41e874-029c-4221-879c-04080e84aca0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:05.164557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41e874-029c-4221-879c-04080e84aca0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:05.164845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41e874-029c-4221-879c-04080e84aca0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:05.166219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1392] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jul 19 12:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:05.171845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4054591-379a-489a-8a11-31e380ba659c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:05.172230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4054591-379a-489a-8a11-31e380ba659c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:05.172606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4054591-379a-489a-8a11-31e380ba659c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:05.246763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e4054591-379a-489a-8a11-31e380ba659c None None] MFA Rules not processed for user `ed446bf868f940e48c5adb2fbcd479f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:05.247938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4054591-379a-489a-8a11-31e380ba659c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:05.290695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:05.329172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 705/1393] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jul 19 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:05.337121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41e874-029c-4221-879c-04080e84aca0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:05.338686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e41e874-029c-4221-879c-04080e84aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:05.352763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:05.446261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41e874-029c-4221-879c-04080e84aca0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:05.519916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:05.520549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:05.520835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:05.521117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:05.522748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:05.590166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:05.638671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:05.640121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d64c06-aa63-48e3-a39b-a77c0e107b2b service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:05.658023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:06.262892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e7f5-1a70-4b00-908f-f806e6c19ac1 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.263466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e7f5-1a70-4b00-908f-f806e6c19ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.263466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81e7f5-1a70-4b00-908f-f806e6c19ac1 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.264925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 706/1395] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:06.270907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133772-286f-465d-bad5-609063eb686a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.271421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133772-286f-465d-bad5-609063eb686a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.271515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133772-286f-465d-bad5-609063eb686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.348021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-04133772-286f-465d-bad5-609063eb686a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:06.349370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04133772-286f-465d-bad5-609063eb686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:06.376279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443efc6-07b8-42a5-ba40-b0b64602d26d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.376656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443efc6-07b8-42a5-ba40-b0b64602d26d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.377008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443efc6-07b8-42a5-ba40-b0b64602d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.441647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3443efc6-07b8-42a5-ba40-b0b64602d26d None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:06.442958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3443efc6-07b8-42a5-ba40-b0b64602d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:06.450520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:24:06.495550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04133772-286f-465d-bad5-609063eb686a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:06.497886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04133772-286f-465d-bad5-609063eb686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:06.523759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:06.535753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 707/1397] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:24:06.580388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0eba6-d385-40ee-8851-609756acd18e None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.580709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0eba6-d385-40ee-8851-609756acd18e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.580981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0eba6-d385-40ee-8851-609756acd18e None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.582488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 708/1398] 10.222.0.13 () {54 vars in 943 bytes} [Tue Jul 19 12:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:24:06.588565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.588859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.589141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.624577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04133772-286f-465d-bad5-609063eb686a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:06.662917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:06.672933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] MFA Rules not processed for user `e2205868e3b04113b500a67eb814e58a`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:06.674501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:06.745668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:06.746469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.746759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.747075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.748781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:06.766767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:06.789499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 709/1399] 10.222.0.13 () {58 vars in 1040 bytes} [Tue Jul 19 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:24:06.800202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:06.811083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaf76a-a01c-421c-b3d6-2eb0450e2195 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:06.844831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:06.846513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8377c8-4032-44a4-b059-b36d3c3ec38e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:06.865257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1400] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:24:06.922976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:06.923713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:06.924007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:06.924325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:06.926603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:06.930390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:07.024121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:07.025905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c30fe6e-289e-402e-8230-f4a50f9a469b service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:07.035614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 710/1401] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:24:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 12:24:09.005834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e00a3-22bb-4863-88f3-d674f5ad05fb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:09.006397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e00a3-22bb-4863-88f3-d674f5ad05fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:09.006397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e00a3-22bb-4863-88f3-d674f5ad05fb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:09.007786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1402] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:09.013054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27f64-a18c-4869-a3ab-72d5f29f7fdc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:09.013479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27f64-a18c-4869-a3ab-72d5f29f7fdc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:09.013628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27f64-a18c-4869-a3ab-72d5f29f7fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:09.082120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8cf27f64-a18c-4869-a3ab-72d5f29f7fdc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:09.083438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8cf27f64-a18c-4869-a3ab-72d5f29f7fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:09.187010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:09.193694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 711/1403] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:09.238412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e00a3-22bb-4863-88f3-d674f5ad05fb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:09.241048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-767e00a3-22bb-4863-88f3-d674f5ad05fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:09.409899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:09.413349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e00a3-22bb-4863-88f3-d674f5ad05fb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:09.554712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:09.555697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:09.556065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:09.556442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:09.558195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:09.671643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:09.673419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9e6fee-1fde-41bd-b4fd-63e436fd3e78 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:09.693177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1404] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:11.783283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c721cd-55ce-4d16-8ec6-65b1271cfa38 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:11.783831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c721cd-55ce-4d16-8ec6-65b1271cfa38 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:11.783831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c721cd-55ce-4d16-8ec6-65b1271cfa38 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:11.785484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 712/1405] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:11.790907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d53be-0d86-48eb-aa5e-0cadf1993eee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:11.791742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d53be-0d86-48eb-aa5e-0cadf1993eee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:11.792208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d53be-0d86-48eb-aa5e-0cadf1993eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:11.795612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:11.868883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb2d53be-0d86-48eb-aa5e-0cadf1993eee None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:11.870033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb2d53be-0d86-48eb-aa5e-0cadf1993eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:11.978403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1406] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:24:12.030535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c721cd-55ce-4d16-8ec6-65b1271cfa38 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:12.032057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e1c721cd-55ce-4d16-8ec6-65b1271cfa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:12.201383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c721cd-55ce-4d16-8ec6-65b1271cfa38 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:12.300095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:12.368467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:12.369224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:12.369627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:12.369974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:12.371708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:12.470970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:12.472561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acf1aa-de82-41ab-8b01-6421501c1656 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:12.492642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 713/1407] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:21.976596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=64 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:22.490053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=64 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:29.533277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d20388-4b1b-49f4-8c2f-f3841463bfd8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:29.533804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d20388-4b1b-49f4-8c2f-f3841463bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:29.533804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d20388-4b1b-49f4-8c2f-f3841463bfd8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:29.535128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1408] 10.222.0.13 () {54 vars in 918 bytes} [Tue Jul 19 12:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:29.540444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa7a4-9a35-417e-9f45-4c4d2a018ec3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:29.540857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa7a4-9a35-417e-9f45-4c4d2a018ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:29.541012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaa7a4-9a35-417e-9f45-4c4d2a018ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:29.616860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-14eaa7a4-9a35-417e-9f45-4c4d2a018ec3 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:29.618053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-14eaa7a4-9a35-417e-9f45-4c4d2a018ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:29.724900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 714/1409] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Jul 19 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:29.739558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940588-b109-4051-a33f-b7eeee60325b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:29.739875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940588-b109-4051-a33f-b7eeee60325b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:29.740162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940588-b109-4051-a33f-b7eeee60325b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:29.741587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1410] 10.222.0.13 () {54 vars in 918 bytes} [Tue Jul 19 12:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:29.746244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829af88b-ab57-41ce-95be-4dff9a0cdec8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:29.746536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829af88b-ab57-41ce-95be-4dff9a0cdec8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:29.746818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829af88b-ab57-41ce-95be-4dff9a0cdec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:29.817335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:29.820031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-829af88b-ab57-41ce-95be-4dff9a0cdec8 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:29.821942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-829af88b-ab57-41ce-95be-4dff9a0cdec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:29.937232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 715/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Jul 19 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:32.467719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5931-68fc-47ea-affe-726bf2d744bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:32.468262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5931-68fc-47ea-affe-726bf2d744bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:32.468262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5931-68fc-47ea-affe-726bf2d744bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:32.469580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1412] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:32.474696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cb2c7-8e0b-4ad1-88a3-4cde9fb7ba43 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:32.475100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cb2c7-8e0b-4ad1-88a3-4cde9fb7ba43 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:32.475266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6cb2c7-8e0b-4ad1-88a3-4cde9fb7ba43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:32.540191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-be6cb2c7-8e0b-4ad1-88a3-4cde9fb7ba43 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:32.541394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-be6cb2c7-8e0b-4ad1-88a3-4cde9fb7ba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:32.595332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:32.639422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 716/1413] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:32.682569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5931-68fc-47ea-affe-726bf2d744bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:32.684198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f71a5931-68fc-47ea-affe-726bf2d744bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:32.828947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5931-68fc-47ea-affe-726bf2d744bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:32.901190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:32.942721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:32.943373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:32.943664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:32.943952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:32.945645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:33.037533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:33.039144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74c916-a65c-4d8d-84d3-3ad458bb6c49 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:33.057154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1414] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:33.952935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c3dbf-d552-4cb9-b3f3-90d844cb2ad0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:33.953473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c3dbf-d552-4cb9-b3f3-90d844cb2ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:33.953473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c3dbf-d552-4cb9-b3f3-90d844cb2ad0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:33.954777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 717/1415] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:33.959738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109dcf1-9166-4125-be28-8ee9d0a55970 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:33.960208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109dcf1-9166-4125-be28-8ee9d0a55970 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:33.960321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109dcf1-9166-4125-be28-8ee9d0a55970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:34.033896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8109dcf1-9166-4125-be28-8ee9d0a55970 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:34.035096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8109dcf1-9166-4125-be28-8ee9d0a55970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:34.061127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:34.129878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1416] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:34.281184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c3dbf-d552-4cb9-b3f3-90d844cb2ad0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:34.282748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-342c3dbf-d552-4cb9-b3f3-90d844cb2ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:34.426883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c3dbf-d552-4cb9-b3f3-90d844cb2ad0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:34.466514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:34.538222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:34.538794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:34.539079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:34.539357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:34.541017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:34.628215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:34.629829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43892582-e525-4697-9348-3bb857fb6ea4 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:34.647609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 718/1417] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:35.525000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac150d-4de6-4d35-a879-d0cd362d0c16 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:35.525556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac150d-4de6-4d35-a879-d0cd362d0c16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:35.525556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac150d-4de6-4d35-a879-d0cd362d0c16 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:35.527003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1418] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:35.531905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bc422-d7cf-44a2-81aa-6d197d0ca0e3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:35.532359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bc422-d7cf-44a2-81aa-6d197d0ca0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:35.532456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bc422-d7cf-44a2-81aa-6d197d0ca0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:35.584961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:35.606930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-302bc422-d7cf-44a2-81aa-6d197d0ca0e3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:35.608138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-302bc422-d7cf-44a2-81aa-6d197d0ca0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:35.704062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 719/1419] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:35.853920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac150d-4de6-4d35-a879-d0cd362d0c16 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:35.855882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0ac150d-4de6-4d35-a879-d0cd362d0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:35.998560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac150d-4de6-4d35-a879-d0cd362d0c16 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:36.010431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:36.111476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:36.112038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:36.112351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:36.112628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:36.114248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:36.205208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:36.206811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea9ff5-2b23-4d0c-9d01-2721e891a9cf service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:36.225136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1420] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:37.081598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f1db-8aee-49e8-8711-224e396c8b39 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:37.082185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f1db-8aee-49e8-8711-224e396c8b39 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:37.082185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120f1db-8aee-49e8-8711-224e396c8b39 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:37.083532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 720/1421] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:37.088405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6d4e-6f3e-448a-8005-b4f070053ac7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:37.088780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6d4e-6f3e-448a-8005-b4f070053ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:37.088970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6d4e-6f3e-448a-8005-b4f070053ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:37.107565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:37.162860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-253d6d4e-6f3e-448a-8005-b4f070053ac7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:37.164065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-253d6d4e-6f3e-448a-8005-b4f070053ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:37.257405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1422] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:37.412968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f1db-8aee-49e8-8711-224e396c8b39 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:37.414425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4120f1db-8aee-49e8-8711-224e396c8b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:37.524236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:37.560054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120f1db-8aee-49e8-8711-224e396c8b39 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:37.680284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:37.680877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:37.681164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:37.681443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:37.683089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:37.759187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:37.773169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:37.774788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a62948-7ea2-4d97-84e1-bd15a33f8b62 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:37.792408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 721/1423] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:38.653059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a46a2a-3d00-4c99-b393-e3132d243294 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:38.653549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a46a2a-3d00-4c99-b393-e3132d243294 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:38.653659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a46a2a-3d00-4c99-b393-e3132d243294 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:38.655013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1424] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:38.660038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f54f26-c8e7-4b11-b3c0-38e151eabdbb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:38.660459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f54f26-c8e7-4b11-b3c0-38e151eabdbb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:38.660644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f54f26-c8e7-4b11-b3c0-38e151eabdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:38.734109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-25f54f26-c8e7-4b11-b3c0-38e151eabdbb None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:38.735310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-25f54f26-c8e7-4b11-b3c0-38e151eabdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:38.831940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 722/1425] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:38.981103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a46a2a-3d00-4c99-b393-e3132d243294 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:38.982630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8a46a2a-3d00-4c99-b393-e3132d243294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:39.067227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:39.129593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a46a2a-3d00-4c99-b393-e3132d243294 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:39.240875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:39.241451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:39.241738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:39.242017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:39.243672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:39.291609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:39.332304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:39.333910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585b192-bbbd-4827-984a-b2631bc010f7 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:39.352171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1426] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:40.151480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56248efa-e28b-4a6d-908e-2ea1ebaa0a39 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:40.152068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56248efa-e28b-4a6d-908e-2ea1ebaa0a39 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:40.152068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56248efa-e28b-4a6d-908e-2ea1ebaa0a39 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:40.153466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 723/1427] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:40.158884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bea58f-884c-4cff-a99e-678d66f69b9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:40.159279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bea58f-884c-4cff-a99e-678d66f69b9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:40.159449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bea58f-884c-4cff-a99e-678d66f69b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:40.233386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41bea58f-884c-4cff-a99e-678d66f69b9c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:40.234593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41bea58f-884c-4cff-a99e-678d66f69b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:40.331343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1428] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:40.483479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56248efa-e28b-4a6d-908e-2ea1ebaa0a39 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:40.485041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-56248efa-e28b-4a6d-908e-2ea1ebaa0a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:40.533256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:40.631645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56248efa-e28b-4a6d-908e-2ea1ebaa0a39 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:40.746918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:40.747482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:40.747765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:40.748041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:40.749707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:40.756503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:40.837487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:40.839103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b1a3a-f89f-4b03-bd80-9935ac790d8d service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:40.857585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 724/1429] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:41.663768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ea02-793f-415b-8689-ef2487ac89cc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:41.664242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ea02-793f-415b-8689-ef2487ac89cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:41.664358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ea02-793f-415b-8689-ef2487ac89cc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:41.665750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1430] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:41.671113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598223-d3fd-4d5b-92ef-efac90a56ff2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:41.671553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598223-d3fd-4d5b-92ef-efac90a56ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:41.671683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba598223-d3fd-4d5b-92ef-efac90a56ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:41.745464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ba598223-d3fd-4d5b-92ef-efac90a56ff2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:41.747248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ba598223-d3fd-4d5b-92ef-efac90a56ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:41.813707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:41.842017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 725/1431] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:41.992689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ea02-793f-415b-8689-ef2487ac89cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:41.994176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8480ea02-793f-415b-8689-ef2487ac89cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:42.011220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:42.141922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ea02-793f-415b-8689-ef2487ac89cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:42.239341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:42.266311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ccac00-112a-409b-8254-261993943d01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:42.266895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccac00-112a-409b-8254-261993943d01 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:42.267182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccac00-112a-409b-8254-261993943d01 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:42.267463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ccac00-112a-409b-8254-261993943d01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:42.269131 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42ccac00-112a-409b-8254-261993943d01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:42.366599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccac00-112a-409b-8254-261993943d01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:42.369069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ccac00-112a-409b-8254-261993943d01 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:42.388714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1432] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:43.186606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9d645-5a0f-4b7e-bc90-bc479ed62743 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:43.187169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9d645-5a0f-4b7e-bc90-bc479ed62743 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:43.187169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9d645-5a0f-4b7e-bc90-bc479ed62743 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:43.188571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 726/1433] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:43.193516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28918cb9-4729-4356-9f16-4b0c20f69e8c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:43.193957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28918cb9-4729-4356-9f16-4b0c20f69e8c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:43.194090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28918cb9-4729-4356-9f16-4b0c20f69e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:43.273293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28918cb9-4729-4356-9f16-4b0c20f69e8c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:43.274518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28918cb9-4729-4356-9f16-4b0c20f69e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:43.315631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:43.373659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1434] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:43.522367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9d645-5a0f-4b7e-bc90-bc479ed62743 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:43.523897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39a9d645-5a0f-4b7e-bc90-bc479ed62743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:43.673251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9d645-5a0f-4b7e-bc90-bc479ed62743 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:43.729193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:43.788509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:43.789084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:43.789372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:43.789653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:43.791303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:43.879125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:43.880757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60512b0-3f22-47e0-bfe8-05b9eec6589c service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:43.898737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 727/1435] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:44.706909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25100d-5a44-4825-a85d-6df28b6c1f2e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:44.707481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25100d-5a44-4825-a85d-6df28b6c1f2e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:44.707481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25100d-5a44-4825-a85d-6df28b6c1f2e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:44.708888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1436] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:44.713919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7e3f9-debe-4863-b969-5fb4470af2db None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:44.714340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7e3f9-debe-4863-b969-5fb4470af2db None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:44.714513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7e3f9-debe-4863-b969-5fb4470af2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:44.789170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d1f7e3f9-debe-4863-b969-5fb4470af2db None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:44.790371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d1f7e3f9-debe-4863-b969-5fb4470af2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:44.792394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:44.887873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 728/1437] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:45.037277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25100d-5a44-4825-a85d-6df28b6c1f2e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:45.038822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d25100d-5a44-4825-a85d-6df28b6c1f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:45.187586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25100d-5a44-4825-a85d-6df28b6c1f2e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:45.213477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:45.299012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:45.299659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:45.299943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:45.300247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:45.301869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:45.405309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:45.406901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b70815-5b52-4e9a-80d7-7c3cbfa5797b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:45.424912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1438] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:24:47.367743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea169c-e1eb-44e3-a066-0c0f8ade809c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:47.368289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea169c-e1eb-44e3-a066-0c0f8ade809c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:47.368416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea169c-e1eb-44e3-a066-0c0f8ade809c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:47.369729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 729/1439] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:47.375520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3effae9-1aa8-48f0-98e6-ca0154c78a1b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:47.375930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3effae9-1aa8-48f0-98e6-ca0154c78a1b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:47.376118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3effae9-1aa8-48f0-98e6-ca0154c78a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:47.414705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:47.440243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3effae9-1aa8-48f0-98e6-ca0154c78a1b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:47.441475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3effae9-1aa8-48f0-98e6-ca0154c78a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:47.534193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1440] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:47.574146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea169c-e1eb-44e3-a066-0c0f8ade809c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:47.575694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6aea169c-e1eb-44e3-a066-0c0f8ade809c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:47.679599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea169c-e1eb-44e3-a066-0c0f8ade809c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:47.711332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:47.752216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:47.752863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:47.753143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:47.753415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:47.755037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:47.860792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:47.862166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0fd3c-37dd-4518-8640-e0e433765f6a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:47.880444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 730/1441] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:50.042010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8002-afc6-4525-a822-97fb33a2c3e2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:50.042649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8002-afc6-4525-a822-97fb33a2c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:50.042649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8002-afc6-4525-a822-97fb33a2c3e2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:50.044492 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1442] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:50.049625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f0b35-6ae2-47f9-8967-922a0ff0b498 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:50.050029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f0b35-6ae2-47f9-8967-922a0ff0b498 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:50.050186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f0b35-6ae2-47f9-8967-922a0ff0b498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:50.113220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dd1f0b35-6ae2-47f9-8967-922a0ff0b498 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:50.114393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dd1f0b35-6ae2-47f9-8967-922a0ff0b498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:50.127592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:50.211635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 731/1443] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:50.250747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd8002-afc6-4525-a822-97fb33a2c3e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:50.252297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fedd8002-afc6-4525-a822-97fb33a2c3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:50.352486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd8002-afc6-4525-a822-97fb33a2c3e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:50.365772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:50.426603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:50.427249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:50.427530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:50.427802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:50.429445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:50.534704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:50.536119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0de5e-8979-41f9-864f-eb0285b64195 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:50.554045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1444] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:52.884069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9dbc1-bb76-428c-91f9-f1dcca192b31 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:52.884608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9dbc1-bb76-428c-91f9-f1dcca192b31 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:52.884608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9dbc1-bb76-428c-91f9-f1dcca192b31 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:52.885965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 732/1445] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:52.891379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151ad5b-a16b-46d0-9ae5-47db6a472260 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:52.891762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151ad5b-a16b-46d0-9ae5-47db6a472260 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:52.891947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151ad5b-a16b-46d0-9ae5-47db6a472260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:52.939011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:52.957043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c151ad5b-a16b-46d0-9ae5-47db6a472260 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:52.958234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c151ad5b-a16b-46d0-9ae5-47db6a472260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:53.055146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1446] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:24:53.094096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9dbc1-bb76-428c-91f9-f1dcca192b31 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:53.095625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7e9dbc1-bb76-428c-91f9-f1dcca192b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:53.198809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9dbc1-bb76-428c-91f9-f1dcca192b31 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:53.238635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:53.272406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:53.273051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:53.273331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:53.273604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:53.275230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:53.410160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:53.411611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb3c72-9200-430c-9b8f-7572ccd30ead service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:53.429929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 733/1447] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:55.564619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456b207-3a69-44bd-a2b3-8b2d510be6c6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:55.565122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456b207-3a69-44bd-a2b3-8b2d510be6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:55.565122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456b207-3a69-44bd-a2b3-8b2d510be6c6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:55.566557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1448] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:55.571833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a20342-d9a8-41af-84ac-3c817736784a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:55.572289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a20342-d9a8-41af-84ac-3c817736784a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:55.572429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a20342-d9a8-41af-84ac-3c817736784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:55.637301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-55a20342-d9a8-41af-84ac-3c817736784a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:55.638498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-55a20342-d9a8-41af-84ac-3c817736784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:55.658212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:55.732799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 734/1449] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:55.772638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456b207-3a69-44bd-a2b3-8b2d510be6c6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:55.774171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c456b207-3a69-44bd-a2b3-8b2d510be6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:55.877629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456b207-3a69-44bd-a2b3-8b2d510be6c6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:55.896876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:55.949862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:55.950498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:55.950783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:55.951059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:55.952711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:56.063187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:56.064668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c18e8a3-d54b-4437-8254-91ab52c8efc7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:56.082476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1450] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:24:58.268932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d46d6-2925-4663-b632-376dc95e16ff None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:58.269495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d46d6-2925-4663-b632-376dc95e16ff None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:58.269495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d46d6-2925-4663-b632-376dc95e16ff None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:58.270880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 735/1451] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:24:58.276277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb696b-309a-44d8-b0a0-e539a8ef73e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:58.276696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb696b-309a-44d8-b0a0-e539a8ef73e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:58.276848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb696b-309a-44d8-b0a0-e539a8ef73e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:58.339899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:58.342614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7aeb696b-309a-44d8-b0a0-e539a8ef73e4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:24:58.344443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7aeb696b-309a-44d8-b0a0-e539a8ef73e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:58.440222 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1452] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:24:58.478331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d46d6-2925-4663-b632-376dc95e16ff None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:24:58.479872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb8d46d6-2925-4663-b632-376dc95e16ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:58.581082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d46d6-2925-4663-b632-376dc95e16ff None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:24:58.630402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:24:58.655674 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:24:58.656361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:24:58.656657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:24:58.656938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:24:58.658638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:24:58.767193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:24:58.768676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a0f98-1610-46cd-b6a4-39c346a39e2b service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:24:58.786705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 736/1453] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:00.866957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb38303-5567-4800-8296-b85892bd87eb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:00.867512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb38303-5567-4800-8296-b85892bd87eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:00.867512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb38303-5567-4800-8296-b85892bd87eb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:00.868841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1454] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:00.874621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50767-8e3a-4440-a8d1-6f5539ffc288 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:00.875006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50767-8e3a-4440-a8d1-6f5539ffc288 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:00.875180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50767-8e3a-4440-a8d1-6f5539ffc288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:00.942997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7fa50767-8e3a-4440-a8d1-6f5539ffc288 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:00.944214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7fa50767-8e3a-4440-a8d1-6f5539ffc288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:00.972962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:01.039827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 737/1455] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:01.079284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb38303-5567-4800-8296-b85892bd87eb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:01.080918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-abb38303-5567-4800-8296-b85892bd87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:01.183251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb38303-5567-4800-8296-b85892bd87eb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:01.211115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:01.256082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:01.256781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:01.257071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:01.257351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:01.258988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:01.365521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:01.367233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926a9d4-cc48-47d6-b5ab-43a93fe57230 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:01.387042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1456] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:03.481422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015da32-96e4-4c1c-95c4-a92df28d0e61 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:03.481975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015da32-96e4-4c1c-95c4-a92df28d0e61 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:03.481975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015da32-96e4-4c1c-95c4-a92df28d0e61 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:03.483351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 738/1457] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:03.488624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c965-0979-4662-a269-99fab05dbee7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:03.489063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c965-0979-4662-a269-99fab05dbee7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:03.489233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c965-0979-4662-a269-99fab05dbee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:03.553281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ea25c965-0979-4662-a269-99fab05dbee7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:03.554430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea25c965-0979-4662-a269-99fab05dbee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:03.563182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:03.653465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1458] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:03.691454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015da32-96e4-4c1c-95c4-a92df28d0e61 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:03.693076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7015da32-96e4-4c1c-95c4-a92df28d0e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:03.797162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015da32-96e4-4c1c-95c4-a92df28d0e61 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:03.852616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:03.871547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:03.872223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:03.872515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:03.872795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:03.874427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:03.979947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:03.982009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535066c-2922-44ad-995b-d2ff303adedf service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:03.999653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 739/1459] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:06.096541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b209ca1-e2f3-4fbd-a34b-47a069d85416 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:06.097103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b209ca1-e2f3-4fbd-a34b-47a069d85416 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:06.097103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b209ca1-e2f3-4fbd-a34b-47a069d85416 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:06.098505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1460] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:06.103991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cb9af-eadd-47bb-9b92-459c8231b71c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:06.104452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cb9af-eadd-47bb-9b92-459c8231b71c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:06.104592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cb9af-eadd-47bb-9b92-459c8231b71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:06.169623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-db8cb9af-eadd-47bb-9b92-459c8231b71c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:06.170803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db8cb9af-eadd-47bb-9b92-459c8231b71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:06.209034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:06.268110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 740/1461] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:06.306163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b209ca1-e2f3-4fbd-a34b-47a069d85416 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:06.307743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5b209ca1-e2f3-4fbd-a34b-47a069d85416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:06.412584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b209ca1-e2f3-4fbd-a34b-47a069d85416 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:06.450417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:06.486381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:06.487028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:06.487319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:06.487604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:06.489335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:06.597002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:06.598433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75512ca3-a261-473e-aabc-ec7e594721db service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:06.617254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1462] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:08.661125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19f8d-9179-4371-8977-bdf9239313a8 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:08.661618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19f8d-9179-4371-8977-bdf9239313a8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:08.661715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe19f8d-9179-4371-8977-bdf9239313a8 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:08.663047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 741/1463] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:08.668370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4efcc-37cd-4357-ade8-5dcb280d25fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:08.668790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4efcc-37cd-4357-ade8-5dcb280d25fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:08.668949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4efcc-37cd-4357-ade8-5dcb280d25fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:08.736654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99d4efcc-37cd-4357-ade8-5dcb280d25fc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:08.737876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99d4efcc-37cd-4357-ade8-5dcb280d25fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:08.762329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:08.854366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1464] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:08.891786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19f8d-9179-4371-8977-bdf9239313a8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:08.893344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7fe19f8d-9179-4371-8977-bdf9239313a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:08.993746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe19f8d-9179-4371-8977-bdf9239313a8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:09.059209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:09.066294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:09.066944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:09.067227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:09.067503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:09.069271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:09.177374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:09.178733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba185aa-db8e-4bf8-94d9-2d275b279d45 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:09.196828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 742/1465] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:11.223217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc51d6-bad9-4c2a-9053-9d663fcc0f25 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:11.223724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc51d6-bad9-4c2a-9053-9d663fcc0f25 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:11.223724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc51d6-bad9-4c2a-9053-9d663fcc0f25 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:11.225112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1466] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:11.230422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4b7c9-177d-4af6-9844-46e1c139c9c3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:11.231526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4b7c9-177d-4af6-9844-46e1c139c9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:11.231814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4b7c9-177d-4af6-9844-46e1c139c9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:11.295087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7ec4b7c9-177d-4af6-9844-46e1c139c9c3 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:11.296720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7ec4b7c9-177d-4af6-9844-46e1c139c9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:11.343861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:11.393613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 743/1467] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:11.431002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc51d6-bad9-4c2a-9053-9d663fcc0f25 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:11.432582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94bc51d6-bad9-4c2a-9053-9d663fcc0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:11.538751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc51d6-bad9-4c2a-9053-9d663fcc0f25 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:11.584341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:11.611884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:11.612546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:11.612833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:11.613111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:11.614737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:11.721089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:11.722521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da4b17-6e34-462c-8f42-bc9089866a5f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:11.740090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1468] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:21.391722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=22 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:21.737791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=66 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:31.452767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64239550-1354-451d-8fe7-53aaeb8fecd6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:31.453312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64239550-1354-451d-8fe7-53aaeb8fecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:31.453312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64239550-1354-451d-8fe7-53aaeb8fecd6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:31.454675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 744/1469] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:31.459778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d922084-d674-4cf4-b4a9-6f5681048447 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:31.460215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d922084-d674-4cf4-b4a9-6f5681048447 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:31.460399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d922084-d674-4cf4-b4a9-6f5681048447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:31.524183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d922084-d674-4cf4-b4a9-6f5681048447 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:31.525404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d922084-d674-4cf4-b4a9-6f5681048447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:31.628665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1470] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:31.710284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f4f97-4f3c-46c7-a118-5bc9602df3ea None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:31.710570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f4f97-4f3c-46c7-a118-5bc9602df3ea None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:31.710846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f4f97-4f3c-46c7-a118-5bc9602df3ea None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:31.712207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 745/1471] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jul 19 12:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:31.717547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab7cd9-3706-41fb-946e-69256ee62510 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:31.717847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab7cd9-3706-41fb-946e-69256ee62510 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:31.718132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab7cd9-3706-41fb-946e-69256ee62510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:31.792188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1eab7cd9-3706-41fb-946e-69256ee62510 None None] MFA Rules not processed for user `ed446bf868f940e48c5adb2fbcd479f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:31.793430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1eab7cd9-3706-41fb-946e-69256ee62510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:31.813199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:31.868815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1472] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jul 19 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:31.876367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f4f97-4f3c-46c7-a118-5bc9602df3ea None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:31.877923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cb3f4f97-4f3c-46c7-a118-5bc9602df3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:31.981739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f4f97-4f3c-46c7-a118-5bc9602df3ea None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:32.052787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:32.053648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:32.053869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:32.054410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:32.055615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:32.109689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:32.164943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:32.166329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b99927-5c7d-4c11-87b5-217cba3eee08 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:32.185114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 746/1473] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:34.313222 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea55645-37b7-4471-9638-317f87f1f6ab None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:34.313793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea55645-37b7-4471-9638-317f87f1f6ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:34.313793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea55645-37b7-4471-9638-317f87f1f6ab None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:34.315145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1474] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:34.320544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aafd7f-44d2-4d3e-8d13-45b3f8ce22d1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:34.320957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aafd7f-44d2-4d3e-8d13-45b3f8ce22d1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:34.321104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aafd7f-44d2-4d3e-8d13-45b3f8ce22d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:34.386505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-06aafd7f-44d2-4d3e-8d13-45b3f8ce22d1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:34.387706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-06aafd7f-44d2-4d3e-8d13-45b3f8ce22d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:34.482288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 747/1475] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:34.505361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea55645-37b7-4471-9638-317f87f1f6ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:34.506860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5ea55645-37b7-4471-9638-317f87f1f6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:34.637708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea55645-37b7-4471-9638-317f87f1f6ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:34.685523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:34.729163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:34.729819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:34.730104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:34.730382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:34.731748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:34.734610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:25:34.734610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:25:34.734610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:25:34.735771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbee19a-fd6a-4d20-9f45-8c1a9720b4dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:34.754437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1476] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Jul 19 12:25:34 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:25:36.810053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae4750-99cc-4a5e-bbfc-0f90c912e2f4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:36.810621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae4750-99cc-4a5e-bbfc-0f90c912e2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:36.810723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae4750-99cc-4a5e-bbfc-0f90c912e2f4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:36.812112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 748/1477] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:36.817243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b8372-c65b-4aa8-a59b-46cce89e8163 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:36.817665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b8372-c65b-4aa8-a59b-46cce89e8163 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:36.817803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b8372-c65b-4aa8-a59b-46cce89e8163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:36.881534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-da9b8372-c65b-4aa8-a59b-46cce89e8163 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:36.882695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da9b8372-c65b-4aa8-a59b-46cce89e8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:36.977718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1478] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:37.021967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae4750-99cc-4a5e-bbfc-0f90c912e2f4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:37.023551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7aae4750-99cc-4a5e-bbfc-0f90c912e2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:37.034019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:37.170286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae4750-99cc-4a5e-bbfc-0f90c912e2f4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:37.248262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:37.281181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:37.281832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:37.282114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:37.282391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:37.284018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:37.373611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:37.375320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c456b8-c798-40b9-b953-a6eeec7aae4f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:37.396189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 749/1479] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:39.635342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61682a55-de25-4d5d-a6b3-8c54ea233b6c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:39.635828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61682a55-de25-4d5d-a6b3-8c54ea233b6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:39.635945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61682a55-de25-4d5d-a6b3-8c54ea233b6c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:39.637908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1480] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:39.643170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196407f3-53e8-4e1f-8f22-8e38a67b3030 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:39.643585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196407f3-53e8-4e1f-8f22-8e38a67b3030 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:39.643736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196407f3-53e8-4e1f-8f22-8e38a67b3030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:39.707168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-196407f3-53e8-4e1f-8f22-8e38a67b3030 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:39.708383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-196407f3-53e8-4e1f-8f22-8e38a67b3030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:39.753683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:39.803268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 750/1481] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:39.840696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61682a55-de25-4d5d-a6b3-8c54ea233b6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:39.842226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-61682a55-de25-4d5d-a6b3-8c54ea233b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:39.865600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:39.947168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61682a55-de25-4d5d-a6b3-8c54ea233b6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:40.020354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:40.020996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:40.021280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:40.021556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:40.023194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:40.094760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:40.130694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:40.132095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f38d19-c08d-46e4-9b6f-11411e2e67d3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:40.149913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1482] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:42.237413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc3d5e-d16f-4996-9822-348d855a15d6 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:42.237927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc3d5e-d16f-4996-9822-348d855a15d6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:42.238425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc3d5e-d16f-4996-9822-348d855a15d6 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:42.240589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 751/1483] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:42.246466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3459e-5528-4fa0-aa72-5fc6a86070a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:42.246933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3459e-5528-4fa0-aa72-5fc6a86070a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:42.247040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d3459e-5528-4fa0-aa72-5fc6a86070a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:42.311090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39d3459e-5528-4fa0-aa72-5fc6a86070a0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:42.312253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39d3459e-5528-4fa0-aa72-5fc6a86070a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:42.408847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1484] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:42.446526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc3d5e-d16f-4996-9822-348d855a15d6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:42.448044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-03fc3d5e-d16f-4996-9822-348d855a15d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:42.552119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc3d5e-d16f-4996-9822-348d855a15d6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:42.634512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:42.636441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:42.637778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:42.638254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:42.638590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:42.640637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:42.748848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:42.750307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571b4a3-75c0-4b40-a89f-eddee3de70a6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:42.768698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 752/1485] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:44.884089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b43ae-794a-42ac-b854-c8b39b973320 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:44.884687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b43ae-794a-42ac-b854-c8b39b973320 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:44.884687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b43ae-794a-42ac-b854-c8b39b973320 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:44.885994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1486] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:44.891866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988263bd-4f91-4bb3-af5a-a0001edd28b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:44.892314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988263bd-4f91-4bb3-af5a-a0001edd28b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:44.892462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988263bd-4f91-4bb3-af5a-a0001edd28b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:44.955694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-988263bd-4f91-4bb3-af5a-a0001edd28b1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:44.956897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-988263bd-4f91-4bb3-af5a-a0001edd28b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:45.007993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:45.052323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 753/1487] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:45.093421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b43ae-794a-42ac-b854-c8b39b973320 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:45.094959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-034b43ae-794a-42ac-b854-c8b39b973320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:45.130865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:45.197822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b43ae-794a-42ac-b854-c8b39b973320 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:45.270370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:45.271021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:45.271308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:45.271586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:45.273252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:45.352301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:45.382834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:45.384298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf03d3-f823-45db-8f9d-452d77e5f554 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:45.402045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1488] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:47.611612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48f7de-abb1-42fe-997e-b73abaf890b5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:47.612158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48f7de-abb1-42fe-997e-b73abaf890b5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:47.612158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48f7de-abb1-42fe-997e-b73abaf890b5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:47.613500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 754/1489] 10.222.0.13 () {54 vars in 926 bytes} [Tue Jul 19 12:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:47.619117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995bd3-bf2e-4a69-ac03-d6f59801bc19 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:47.619531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995bd3-bf2e-4a69-ac03-d6f59801bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:47.619686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995bd3-bf2e-4a69-ac03-d6f59801bc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:47.683635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b8995bd3-bf2e-4a69-ac03-d6f59801bc19 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:47.685588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8995bd3-bf2e-4a69-ac03-d6f59801bc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:47.779907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1490] 10.222.0.13 () {58 vars in 1023 bytes} [Tue Jul 19 12:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:47.835035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbc180-667e-438e-85ba-b047236246cc None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:47.835324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbc180-667e-438e-85ba-b047236246cc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:47.835605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedbc180-667e-438e-85ba-b047236246cc None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:47.837001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 755/1491] 10.222.0.13 () {54 vars in 965 bytes} [Tue Jul 19 12:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:25:47.842796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722a43-5aa7-4cd8-a6df-a7045457eaa8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:47.843123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722a43-5aa7-4cd8-a6df-a7045457eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:47.843433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37722a43-5aa7-4cd8-a6df-a7045457eaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:47.893117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:47.919329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-37722a43-5aa7-4cd8-a6df-a7045457eaa8 None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:25:47.920554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37722a43-5aa7-4cd8-a6df-a7045457eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:47.994535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1492] 10.222.0.13 () {58 vars in 1062 bytes} [Tue Jul 19 12:25:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:25:48.003372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbc180-667e-438e-85ba-b047236246cc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:25:48.004934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bedbc180-667e-438e-85ba-b047236246cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:48.106678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedbc180-667e-438e-85ba-b047236246cc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:25:48.178364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:25:48.178940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:25:48.179220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:25:48.179493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:25:48.181147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:25:48.190998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:48.291943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:25:48.293449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137db8e-f687-4f45-8010-5cdf79bb23f0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:25:48.311197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 756/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:25:57.992498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:25:58.309609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=50 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:21.429977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:21.430465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:21.430572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:21.505443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:27:21.506690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:21.602286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1494] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:27:21.610367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:21.610879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:21.610983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:21.684439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:27:21.685545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:21.780819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 757/1495] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:27:21.785462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:21.786655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:21.797856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185ea7e-7b27-4295-96f4-c49e62ec069d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:21.857719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:21.891744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:21.892428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:21.892717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:21.892989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:21.893703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:27:21.894012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:21.895012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:21.895012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:27:21.895012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:27:21.895901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:21.906914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:27:21.915097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:21.924136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d5984-1853-45ae-b73d-2f7aef6ba560 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:21.983799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:22.019537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:22.020232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:22.020524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:22.020802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:22.021498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:22.023280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:22.023280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:27:22.023280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:27:22.024537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:22.080224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 758/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:27:21 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:27:22.084241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:22.085823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:22.123359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1815d7-148f-44f6-b087-5d69118a0ff7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:22.217056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:22.217731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:22.218018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:22.218297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:22.219000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:22.220015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:22.220015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:27:22.220015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:27:22.221321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:22.261961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:27:22 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:27:22.266036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:22.267600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:22.330900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c7ef3-6f23-4d0f-8855-cce88a3bd895 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:22.415156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:22.426563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:22.427232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:22.427514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:22.427788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:22.428544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:22.429565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:22.429565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:27:22.429565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:27:22.430444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:22.442459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 759/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:27:22.446542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:22.448038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:22.459251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba507057-2ee2-41dc-896a-8895044cb205 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:22.484524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:22.553926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:22.554517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:22.554801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:22.555071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:22.555787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:22.556822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:22.556822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:27:22.556822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:27:22.557705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19658e44-8956-4777-8d33-cb6003e0840d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:22.569792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:27:22.574122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:22.575681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:22.609592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d25eaf-b97d-49e3-85c4-6587eaecf7b4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:22.703939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:22.704858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:22.705144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:22.705422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] PATH_INFO: `/v3/projects/ab0e5459cf734c45992f610038b3d1a4/users/b7cadabdc9864590a0df844b09064555/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:22.727482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0e5459cf734c45992f610038b3d1a4, user_id=b7cadabdc9864590a0df844b09064555, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:22.728562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:22.728562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:27:22.728562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:27:22.731113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b34e26-b73e-40e6-a663-f3e2db240fa7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:22.755720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 760/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:27:22 2022] PUT /identity/v3/projects/ab0e5459cf734c45992f610038b3d1a4/users/b7cadabdc9864590a0df844b09064555/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:27:22.768406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fb110-6dc0-4ec2-8002-6c6644963e7f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:22.768703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fb110-6dc0-4ec2-8002-6c6644963e7f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:22.768984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fb110-6dc0-4ec2-8002-6c6644963e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:22.942016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a40fb110-6dc0-4ec2-8002-6c6644963e7f None None] MFA Rules not processed for user `b7cadabdc9864590a0df844b09064555`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:27:22.943118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a40fb110-6dc0-4ec2-8002-6c6644963e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:23.017408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1502] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:27:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:27:23.031067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5b0c-67b3-40c6-a65c-b08714e2f5f2 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:23.031359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5b0c-67b3-40c6-a65c-b08714e2f5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:23.031641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5b0c-67b3-40c6-a65c-b08714e2f5f2 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:23.033031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 761/1503] 10.222.0.13 () {54 vars in 964 bytes} [Tue Jul 19 12:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:27:23.037832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba550e-beb4-4c19-b667-30788f848157 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:23.038119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba550e-beb4-4c19-b667-30788f848157 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:23.038394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba550e-beb4-4c19-b667-30788f848157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:23.067079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:23.110286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-91ba550e-beb4-4c19-b667-30788f848157 None None] MFA Rules not processed for user `a58d153dd25943ef80eea95893118e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:27:23.111472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-91ba550e-beb4-4c19-b667-30788f848157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:23.184566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1504] 10.222.0.13 () {58 vars in 1061 bytes} [Tue Jul 19 12:27:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:27:23.192169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5b0c-67b3-40c6-a65c-b08714e2f5f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:23.193645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cb2f5b0c-67b3-40c6-a65c-b08714e2f5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:23.243306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:23.297625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5b0c-67b3-40c6-a65c-b08714e2f5f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:23.370109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:23.371522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:23.371810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:23.372090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:23.373868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:23.474940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:23.476345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:23.479791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:23.494369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 762/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:27:23.537080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe43a-89e4-4281-9dae-00e070513eeb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:23.537512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe43a-89e4-4281-9dae-00e070513eeb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:23.537620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe43a-89e4-4281-9dae-00e070513eeb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:23.538897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1506] 10.222.0.13 () {52 vars in 829 bytes} [Tue Jul 19 12:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:27:23.542570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:23.544330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:23.762463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f0429-cc5a-4402-97e7-ef96cba2af5b service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:23.853607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:23.854478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:23.854765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:23.855041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] PATH_INFO: `/v3/users/b7cadabdc9864590a0df844b09064555` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:23.867422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cadabdc9864590a0df844b09064555)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:23.868498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:23.868498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:27:23.868498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:27:23.869703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:23.879854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:23.929887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cadabdc9864590a0df844b09064555'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:23.939491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cadabdc9864590a0df844b09064555'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:23.939921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cadabdc9864590a0df844b09064555'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:23.958557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cadabdc9864590a0df844b09064555'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:23.970303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cadabdc9864590a0df844b09064555'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:23.975822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 763/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:27:23 2022] DELETE /identity/v3/users/b7cadabdc9864590a0df844b09064555 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:27:23.980339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:23.980631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:23.980914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:24.035051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:27:24.036850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:24.111536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:24.124646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1508] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:27:24.134216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:24.135426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:24.235696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4851b2-c9f4-4238-bcdb-de608358154a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:24.289048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:24.310585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:24.311234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:24.311518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:24.311792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:24.313440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:24.421055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:24.422516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1598cbe-5e5e-4367-811e-25282b2a5232 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:24.440520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 764/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:27:24.799758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:27:24.801074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:27:24.813094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c09b2-c047-422c-ac4c-206fb73d66a2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:27:24.911444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:27:24.912336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:27:24.912622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:27:24.912896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] PATH_INFO: `/v3/projects/ab0e5459cf734c45992f610038b3d1a4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:27:24.922522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0e5459cf734c45992f610038b3d1a4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:27:24.923540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:27:24.923540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:27:24.923540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:27:24.924798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:27:24.980893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0e5459cf734c45992f610038b3d1a4', 'request_id': 'req-032a51cd-3515-4c75-96c4-7087d626dceb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:24.986821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0e5459cf734c45992f610038b3d1a4', 'request_id': 'req-032a51cd-3515-4c75-96c4-7087d626dceb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:24.997741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0e5459cf734c45992f610038b3d1a4', 'request_id': 'req-032a51cd-3515-4c75-96c4-7087d626dceb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:25.027476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] The token cache is being invalidate because project ab0e5459cf734c45992f610038b3d1a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:27:25.027764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:25.028216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-032a51cd-3515-4c75-96c4-7087d626dceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:27:25.029645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:27:24 2022] DELETE /identity/v3/projects/ab0e5459cf734c45992f610038b3d1a4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:27:34.438174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=62 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:27:35.025495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:29:01.905249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20621d06-eb10-446f-aadd-386b6b0ad906 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:01.905841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20621d06-eb10-446f-aadd-386b6b0ad906 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:01.905841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20621d06-eb10-446f-aadd-386b6b0ad906 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:01.907160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:01.912018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26d1a-eb73-4dce-a9ba-b437b6cace62 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:01.912449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26d1a-eb73-4dce-a9ba-b437b6cace62 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:01.912609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26d1a-eb73-4dce-a9ba-b437b6cace62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:01.990627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40b26d1a-eb73-4dce-a9ba-b437b6cace62 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:01.991836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40b26d1a-eb73-4dce-a9ba-b437b6cace62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:02.123790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:02.278034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20621d06-eb10-446f-aadd-386b6b0ad906 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:02.280716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20621d06-eb10-446f-aadd-386b6b0ad906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:02.406548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20621d06-eb10-446f-aadd-386b6b0ad906 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:02.412415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4a210-ffda-49d0-8ff2-d323c71d100d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:02.413071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4a210-ffda-49d0-8ff2-d323c71d100d service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:02.413355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4a210-ffda-49d0-8ff2-d323c71d100d service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:02.413632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4a210-ffda-49d0-8ff2-d323c71d100d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:02.422720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4a210-ffda-49d0-8ff2-d323c71d100d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:02.424339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4a210-ffda-49d0-8ff2-d323c71d100d service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:02.434061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 766/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:03.257615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199571d-f1c4-4c45-9f79-d939c2285d8e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:03.258188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199571d-f1c4-4c45-9f79-d939c2285d8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:03.258188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199571d-f1c4-4c45-9f79-d939c2285d8e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:03.259579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:03.264472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a346a7c-46b0-49ec-9d73-46d3fb302986 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:03.264898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a346a7c-46b0-49ec-9d73-46d3fb302986 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:03.265032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a346a7c-46b0-49ec-9d73-46d3fb302986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:03.282858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4a346a7c-46b0-49ec-9d73-46d3fb302986 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:03.283963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4a346a7c-46b0-49ec-9d73-46d3fb302986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:03.291663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:03.447744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199571d-f1c4-4c45-9f79-d939c2285d8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:03.467153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199571d-f1c4-4c45-9f79-d939c2285d8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:03.475439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbf632-abee-4c08-9f28-17dadfdcdd17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:03.476078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbf632-abee-4c08-9f28-17dadfdcdd17 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:03.476395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbf632-abee-4c08-9f28-17dadfdcdd17 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:03.476677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbf632-abee-4c08-9f28-17dadfdcdd17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:03.485608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbf632-abee-4c08-9f28-17dadfdcdd17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:03.487574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbf632-abee-4c08-9f28-17dadfdcdd17 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:03.497291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:03.586748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a3daf-65c6-4e57-a715-3d69417557a4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:03.586960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a3daf-65c6-4e57-a715-3d69417557a4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:03.587228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a3daf-65c6-4e57-a715-3d69417557a4 None None] PATH_INFO: `/v3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:03.588567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 768/1517] 10.222.0.42 () {54 vars in 925 bytes} [Tue Jul 19 12:29:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:29:03.594707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5db5e-b16a-4064-9bbb-9fca50a9d4f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:03.595000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5db5e-b16a-4064-9bbb-9fca50a9d4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:03.595280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5db5e-b16a-4064-9bbb-9fca50a9d4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:03.653349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-24b5db5e-b16a-4064-9bbb-9fca50a9d4f5 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:03.655056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-24b5db5e-b16a-4064-9bbb-9fca50a9d4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:03.672086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:29:03.725095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Jul 19 12:29:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:07.440807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d18b2-10ee-4a46-95f8-35b37e481f7a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:07.441264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d18b2-10ee-4a46-95f8-35b37e481f7a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:07.441370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d18b2-10ee-4a46-95f8-35b37e481f7a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:07.442700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 769/1519] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jul 19 12:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:07.447907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cbaef-b035-42f8-bf85-8157bfc8ac99 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:07.448347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cbaef-b035-42f8-bf85-8157bfc8ac99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:07.448496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cbaef-b035-42f8-bf85-8157bfc8ac99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:07.466693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-882cbaef-b035-42f8-bf85-8157bfc8ac99 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:07.467892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-882cbaef-b035-42f8-bf85-8157bfc8ac99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:07.476219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jul 19 12:29:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:07.483418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d18b2-10ee-4a46-95f8-35b37e481f7a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:07.498968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d18b2-10ee-4a46-95f8-35b37e481f7a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:07.503810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971da051-f08d-44e0-acfd-600a6f275c75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:07.504470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971da051-f08d-44e0-acfd-600a6f275c75 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:07.504753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971da051-f08d-44e0-acfd-600a6f275c75 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:07.505028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971da051-f08d-44e0-acfd-600a6f275c75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:07.513061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971da051-f08d-44e0-acfd-600a6f275c75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:07.515263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971da051-f08d-44e0-acfd-600a6f275c75 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:07.524617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 770/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:29:14.729655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc10a86-03df-4bad-ba55-48b33bafcec7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:14.730241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc10a86-03df-4bad-ba55-48b33bafcec7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:14.730241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc10a86-03df-4bad-ba55-48b33bafcec7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:14.731657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:14.736415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393276f7-3385-4d18-9ede-3b1179ee7396 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:14.736912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393276f7-3385-4d18-9ede-3b1179ee7396 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:14.737059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393276f7-3385-4d18-9ede-3b1179ee7396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:14.755519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-393276f7-3385-4d18-9ede-3b1179ee7396 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:14.756752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-393276f7-3385-4d18-9ede-3b1179ee7396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:14.764544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:14.919677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc10a86-03df-4bad-ba55-48b33bafcec7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:14.934598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc10a86-03df-4bad-ba55-48b33bafcec7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:14.939851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101429d5-458d-42a4-ac94-20b00c28c8fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:14.940552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101429d5-458d-42a4-ac94-20b00c28c8fa service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:14.940835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101429d5-458d-42a4-ac94-20b00c28c8fa service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:14.941111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101429d5-458d-42a4-ac94-20b00c28c8fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:14.949015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101429d5-458d-42a4-ac94-20b00c28c8fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:14.950573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101429d5-458d-42a4-ac94-20b00c28c8fa service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:14.959702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:15.771664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f8536-6710-4021-a2e5-ecf7022fca9c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:15.772231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f8536-6710-4021-a2e5-ecf7022fca9c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:15.772345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f8536-6710-4021-a2e5-ecf7022fca9c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:15.773633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 19 12:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:15.778358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c932-1bdd-4469-a9c2-824eab7a2692 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:15.778796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c932-1bdd-4469-a9c2-824eab7a2692 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:15.778899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158c932-1bdd-4469-a9c2-824eab7a2692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:15.796287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2158c932-1bdd-4469-a9c2-824eab7a2692 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:15.797465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2158c932-1bdd-4469-a9c2-824eab7a2692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:15.805632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 19 12:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:15.958821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f8536-6710-4021-a2e5-ecf7022fca9c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:15.973009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f8536-6710-4021-a2e5-ecf7022fca9c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:15.978388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1cc5c-6af2-41c7-b936-21f95c8be917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:15.979020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1cc5c-6af2-41c7-b936-21f95c8be917 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:15.979306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1cc5c-6af2-41c7-b936-21f95c8be917 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:15.979588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1cc5c-6af2-41c7-b936-21f95c8be917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:15.988039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1cc5c-6af2-41c7-b936-21f95c8be917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:15.989664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1cc5c-6af2-41c7-b936-21f95c8be917 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:15.999247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 773/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:19.907037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14ab2c-883b-4f44-8f74-8a9b1506d8bc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:19.907608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14ab2c-883b-4f44-8f74-8a9b1506d8bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:19.907608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14ab2c-883b-4f44-8f74-8a9b1506d8bc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:19.909665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:29:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:19.914867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e05b2d-7a1d-4a69-93ac-d2634834de45 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:19.915325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e05b2d-7a1d-4a69-93ac-d2634834de45 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:19.915429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e05b2d-7a1d-4a69-93ac-d2634834de45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:19.933606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-05e05b2d-7a1d-4a69-93ac-d2634834de45 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:19.934799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-05e05b2d-7a1d-4a69-93ac-d2634834de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:19.943085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:20.015132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14ab2c-883b-4f44-8f74-8a9b1506d8bc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:20.016996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb14ab2c-883b-4f44-8f74-8a9b1506d8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:20.095354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14ab2c-883b-4f44-8f74-8a9b1506d8bc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:20.098886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e056e-2043-4106-94b8-057f0d3f13ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:20.099525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e056e-2043-4106-94b8-057f0d3f13ea service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:20.099809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e056e-2043-4106-94b8-057f0d3f13ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:20.100083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e056e-2043-4106-94b8-057f0d3f13ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:20.109593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e056e-2043-4106-94b8-057f0d3f13ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:20.111027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e056e-2043-4106-94b8-057f0d3f13ea service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:20.120508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:22.155076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6cb9-a4c1-43e7-a1dd-490172db2d32 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:22.155615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6cb9-a4c1-43e7-a1dd-490172db2d32 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:22.155713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6cb9-a4c1-43e7-a1dd-490172db2d32 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:22.157112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:22.162946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651f77a-01df-49e5-9689-7ad4ee6e47b5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:22.163374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651f77a-01df-49e5-9689-7ad4ee6e47b5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:22.163532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651f77a-01df-49e5-9689-7ad4ee6e47b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:22.181584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7651f77a-01df-49e5-9689-7ad4ee6e47b5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:22.182791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7651f77a-01df-49e5-9689-7ad4ee6e47b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:22.191176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:22.228759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6cb9-a4c1-43e7-a1dd-490172db2d32 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:22.230417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6b9c6cb9-a4c1-43e7-a1dd-490172db2d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:22.243247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6cb9-a4c1-43e7-a1dd-490172db2d32 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:22.246764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed56370-f232-48fa-bc68-803bfbb11c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:22.247397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed56370-f232-48fa-bc68-803bfbb11c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:22.247683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed56370-f232-48fa-bc68-803bfbb11c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:22.248254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed56370-f232-48fa-bc68-803bfbb11c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:22.257598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed56370-f232-48fa-bc68-803bfbb11c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:22.259025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed56370-f232-48fa-bc68-803bfbb11c73 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:22.269019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 776/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:25.915498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5af09b-5bfe-466a-b83d-94f659f4bcba None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:25.916059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5af09b-5bfe-466a-b83d-94f659f4bcba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:25.916059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5af09b-5bfe-466a-b83d-94f659f4bcba None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:25.917479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:25.922734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7e707-675c-45e4-a708-c53021a31f59 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:25.923114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7e707-675c-45e4-a708-c53021a31f59 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:25.923293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d7e707-675c-45e4-a708-c53021a31f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:25.930143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:29:25.942498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d3d7e707-675c-45e4-a708-c53021a31f59 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:25.943669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d3d7e707-675c-45e4-a708-c53021a31f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:25.951381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:26.250689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5af09b-5bfe-466a-b83d-94f659f4bcba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:26.252927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab5af09b-5bfe-466a-b83d-94f659f4bcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:26.319553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:29:26.334620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5af09b-5bfe-466a-b83d-94f659f4bcba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:26.338712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d78b6d-ccca-47f9-b0dd-831cd6d6fb93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:26.339549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d78b6d-ccca-47f9-b0dd-831cd6d6fb93 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:26.339846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d78b6d-ccca-47f9-b0dd-831cd6d6fb93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:26.340134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d78b6d-ccca-47f9-b0dd-831cd6d6fb93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:26.349758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d78b6d-ccca-47f9-b0dd-831cd6d6fb93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:26.351335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d78b6d-ccca-47f9-b0dd-831cd6d6fb93 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:26.360693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:28.393983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cd5d-360e-4a76-b29e-5c0bf739b0ad None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:28.394559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cd5d-360e-4a76-b29e-5c0bf739b0ad None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:28.394559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cd5d-360e-4a76-b29e-5c0bf739b0ad None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:28.395918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 19 12:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:29:28.401259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01805dc5-ef51-4c29-ba5c-dc7cb3bc7ff6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:28.401678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01805dc5-ef51-4c29-ba5c-dc7cb3bc7ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:28.401819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01805dc5-ef51-4c29-ba5c-dc7cb3bc7ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:28.419886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-01805dc5-ef51-4c29-ba5c-dc7cb3bc7ff6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:29:28.421107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01805dc5-ef51-4c29-ba5c-dc7cb3bc7ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:28.429419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 19 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:29:28.731877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cd5d-360e-4a76-b29e-5c0bf739b0ad None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:29:28.733562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bf85cd5d-360e-4a76-b29e-5c0bf739b0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:29:28.746599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cd5d-360e-4a76-b29e-5c0bf739b0ad None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:29:28.750359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ee467-e53e-4f1c-a6cf-bd2c46a6f065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:29:28.750992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ee467-e53e-4f1c-a6cf-bd2c46a6f065 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:29:28.751276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ee467-e53e-4f1c-a6cf-bd2c46a6f065 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:29:28.751556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ee467-e53e-4f1c-a6cf-bd2c46a6f065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:29:28.760389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ee467-e53e-4f1c-a6cf-bd2c46a6f065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:29:28.761821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ee467-e53e-4f1c-a6cf-bd2c46a6f065 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:29:28.771564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 779/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:29:38.415233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:29:38.766980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=8 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:57.364017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.365213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.366164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.389337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.390553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.405037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1540] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:57.414255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.414812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.414812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.445239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.446460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.459070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 780/1541] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:57.464381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:57.482910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6e0b0-3168-45da-9228-3fb41885e9c7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:57.492092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:57.492796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.493087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.493371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.494110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:57.494423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:57.496041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:57.513400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:57.522180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:57.531876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73adb-076b-4fc2-adb5-a7b198fdcc00 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:57.549899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2a646-7267-4637-847f-60be1d159914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:57.550599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2a646-7267-4637-847f-60be1d159914 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.550887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2a646-7267-4637-847f-60be1d159914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.551167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2a646-7267-4637-847f-60be1d159914 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.551895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2a646-7267-4637-847f-60be1d159914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:57.553873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2a646-7267-4637-847f-60be1d159914 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:57.609864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 781/1543] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/projects => generated 405 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:57.621053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:57.632971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e7bdf-9142-417a-85f1-67b6748f333a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.633683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e7bdf-9142-417a-85f1-67b6748f333a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.634137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e7bdf-9142-417a-85f1-67b6748f333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.639616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf0a9-834b-406f-a3be-05f1bb0c3013 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:57.669968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c52e7bdf-9142-417a-85f1-67b6748f333a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.671544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c52e7bdf-9142-417a-85f1-67b6748f333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.795358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155e93f-51c7-4acc-b7a1-7a40c13ccf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:57.796037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e93f-51c7-4acc-b7a1-7a40c13ccf9b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.797639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e93f-51c7-4acc-b7a1-7a40c13ccf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.797920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155e93f-51c7-4acc-b7a1-7a40c13ccf9b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.798617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155e93f-51c7-4acc-b7a1-7a40c13ccf9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:57.803031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155e93f-51c7-4acc-b7a1-7a40c13ccf9b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:57.823007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 782/1544] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:30:57.824353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1841d-4405-40a0-8308-ddc939ab873e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.824719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1841d-4405-40a0-8308-ddc939ab873e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.825001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1841d-4405-40a0-8308-ddc939ab873e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.851145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-76b1841d-4405-40a0-8308-ddc939ab873e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.852541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-76b1841d-4405-40a0-8308-ddc939ab873e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.853641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1545] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/users => generated 303 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:57.855059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.855350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.855627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.862450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 783/1546] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:57.864007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c32c5-6da6-4d3b-a0fb-d93f9e6f9cdc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.864350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c32c5-6da6-4d3b-a0fb-d93f9e6f9cdc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.864636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c32c5-6da6-4d3b-a0fb-d93f9e6f9cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.893607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b91c32c5-6da6-4d3b-a0fb-d93f9e6f9cdc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.894741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b91c32c5-6da6-4d3b-a0fb-d93f9e6f9cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.895459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.896772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.912616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 784/1547] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:57.913869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1548] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:57.915471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:57.916151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd6d92-b003-46fc-96c5-365570b1f1aa None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.916523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd6d92-b003-46fc-96c5-365570b1f1aa None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.917212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd6d92-b003-46fc-96c5-365570b1f1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.933636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd56b3c-a593-49ff-b420-6c2d48dcdd95 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:57.938775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc124d23-01ed-4e60-ae11-3f9ce229c094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:57.939538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc124d23-01ed-4e60-ae11-3f9ce229c094 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.939906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc124d23-01ed-4e60-ae11-3f9ce229c094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.940209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc124d23-01ed-4e60-ae11-3f9ce229c094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.941264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc124d23-01ed-4e60-ae11-3f9ce229c094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:57.942647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc124d23-01ed-4e60-ae11-3f9ce229c094 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:57.948807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-dbdd6d92-b003-46fc-96c5-365570b1f1aa None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.950127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dbdd6d92-b003-46fc-96c5-365570b1f1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.954514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1549] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:57.956795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb7493-362b-4a04-a9ab-c170dd2620b5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.957244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb7493-362b-4a04-a9ab-c170dd2620b5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.957840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb7493-362b-4a04-a9ab-c170dd2620b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.962539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 785/1550] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:57.968265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f2f9-3eb4-4b50-9237-8665f10b3d66 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:57.968265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f2f9-3eb4-4b50-9237-8665f10b3d66 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:57.968265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f2f9-3eb4-4b50-9237-8665f10b3d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:57.990617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-becb7493-362b-4a04-a9ab-c170dd2620b5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.991816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-becb7493-362b-4a04-a9ab-c170dd2620b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:57.993703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4907f2f9-3eb4-4b50-9237-8665f10b3d66 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:57.994872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4907f2f9-3eb4-4b50-9237-8665f10b3d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.000264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1551] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.001416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.001706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.001983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.007124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 786/1552] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.012108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.012446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.012731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.021221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.022422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.030559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1553] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.031155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.033360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.037634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.043654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30847cf7-6fe4-4061-aa5e-725c8fa1f532 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.048414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 787/1554] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.049814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.051660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936cc3a-661d-4e12-a4f8-7b84042848de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.052368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936cc3a-661d-4e12-a4f8-7b84042848de admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.052662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936cc3a-661d-4e12-a4f8-7b84042848de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.052936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936cc3a-661d-4e12-a4f8-7b84042848de admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.053658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936cc3a-661d-4e12-a4f8-7b84042848de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.055263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936cc3a-661d-4e12-a4f8-7b84042848de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.059385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa63a-08a0-45b8-873b-109d45bd74b1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.063929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.064615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.065453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.065728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.069298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1555] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:58.069451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.069451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.069633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.069633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.069633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.069864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:30:58.069864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:30:58.069864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:30:58.069864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.081594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 788/1556] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.082177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.085228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.086505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.091702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c58c01-3356-4a5b-a14d-d2fcf2463a96 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.095062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1557] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.096470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.096818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.097453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.097733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.098004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.098715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.099044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.100727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.107221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc81ff8-492a-4c26-8112-13e0d56c81e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.112446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.113095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.113407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.113692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.113819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 789/1558] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.114462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.114594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.114778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.116387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268bc7f-21ce-4762-ad59-388b7947e620 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.135579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1559] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.135789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ae19c-2235-4af7-bf1a-3e9c0bbb4222 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.135904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.135904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.135904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.136228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.136228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.136228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.136228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.136599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.136743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.138318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d4612-ba4b-482b-a799-df6db23dd3ee admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.148360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.149493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.149882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 790/1560] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.151087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.151375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.151648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.162980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1561] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.163754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.173120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfe8b1-a760-4ea5-b193-990c91b183a2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.173987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.175850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.178165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.179029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.179316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.179587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] PATH_INFO: `/v3/projects/74d80918a03945e6adceec468320d033/users/4e42c2ed4d6b460c97cf9f8cf4747c9a/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.184404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 791/1562] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.188313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.194568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d80918a03945e6adceec468320d033, user_id=4e42c2ed4d6b460c97cf9f8cf4747c9a, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.195602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:30:58.195602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:30:58.195602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:30:58.197552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f179-1e61-42fe-9e9f-c70c002e3373 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.197687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61669a-d825-4335-b091-d087e99a2fdc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.202349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.203066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.203279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.203562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.204280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.206219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.211177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1563] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:30:58 2022] PUT /identity/v3/projects/74d80918a03945e6adceec468320d033/users/4e42c2ed4d6b460c97cf9f8cf4747c9a/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:30:58.212253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.212546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.212824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.230655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.231845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.231960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 792/1564] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.235707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.260430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:58.266073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:58.293716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1565] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.297610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.303121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21255461-af0b-4a80-a5ea-ed1f525c8063 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.304137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bf3b4-9e19-4465-aac7-d42e5d6de3a0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.307784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.308482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.308703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.308814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.309024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.309339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.309624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.309733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.309903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.310584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.311592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:30:58.311592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:30:58.311592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:30:58.311858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.312873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44481b1-0ebc-4459-b8b9-397215e0ebd4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.337275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 793/1566] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.338031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1567] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.338440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.342216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.342510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.342795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.360048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.361291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.402973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2340-ef3d-4538-89af-736a7fe92835 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.425041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1568] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.426583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.436073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35141986-b456-403f-8d0f-a7e96f9de25f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.440699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.441340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.441622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.441898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.442615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.442922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.444546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e25c1f-5619-4537-bd56-991fbf8c9557 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.460299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1569] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.465730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.466020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.466302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.467481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.468270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.468556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.468843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.469570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.472147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.484553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:58.485763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:58.494679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1570] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:58.497874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.500324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 794/1571] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.501299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.505997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482fdc9-b637-43ca-a46d-fbaa83dcda6a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.546409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:58.562354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.563022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.563313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.563593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.564316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.566278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.569605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b33c9-7e3a-4da1-86d8-8f3b082cd237 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.574413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.575301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.575431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.575623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.576409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.576714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.578682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.590316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1572] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.590316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.591853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 795/1573] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.592691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.599557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8952c35-2fce-47e5-aec3-250dd49df0c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.603720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2477df-60db-46cd-b3b0-d574896d63e9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.604108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.604807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.605093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.605368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.606149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.608193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.611713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.612430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.612717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.612995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.613685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.614707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:30:58.614707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:30:58.614707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:30:58.615946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.629664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1574] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.630473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.640880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357436d0-e0e0-4b62-922a-6e89f29e90ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.642173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 796/1575] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.643496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.646568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.647215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.647620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.647906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.648662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:30:58.648971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.650562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.653149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:58.655123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc5d7f-40cc-4085-b8f6-fa8d0cb1dc31 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.661311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.661978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.662267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.662548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.663247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.665227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.665620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1576] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:30:58.666550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.681404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce661a-1512-43d4-a676-0caf675c9608 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.686919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.687604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.687916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.688212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.688918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.690979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.697303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 797/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.698278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.719169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1578] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.719169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.728192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e82d5-b3bd-4d8d-9801-7e7ea3d7e7fb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.757097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134db3a3-4147-499a-9de2-35c53c314012 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.760719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.761365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.761701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.761983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.788377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.789233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.789520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.789797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.790515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.792116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.802991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1579] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:58.804020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.815559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09da07-d28e-467c-b32e-f39cbe557047 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.820703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.821439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.821722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.822002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.822750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.823026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.824542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.824733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.834614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 798/1580] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:30:58.835437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.845220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53827a8b-d086-4409-afc3-3b7769d9344a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.850446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.851605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.851605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.851839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.852406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.854026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.854950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1581] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.855785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.864795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 799/1582] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:58.865628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.876155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648a75f-e8e1-4223-9b25-bced7246eb9c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.913177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:58.940597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0c300-f98f-444d-ba9c-681defa9be3b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.945918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.946648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.946903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.947181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.947911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.948249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.948928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.949216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.949496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.949596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.950197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.952124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.961247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1583] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:58.961870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:58.973266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341cbb-d22b-47d3-b304-7e572454e2a6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:58.979214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:58.980014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:58.980393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:58.980682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:58.981414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:58.983670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:58.986846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 800/1584] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:58.987545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.012255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:58 2022] POST /identity/v3/users => generated 284 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:59.014009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.027172 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220e5de-208e-4f49-b9e7-03eeef6ea2ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.058129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:59.068368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dc070-2684-4aa5-815e-8d96d2d7bf8e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.088425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.089138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.089425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.089702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.090437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.091745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.092527 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.092693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.092891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.093304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.094161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.096481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.107498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1586] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.108015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 801/1587] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.108914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.111424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.121435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555f592-f787-45ac-a2ad-24b195a514b6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.122593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a54b7-68f8-431b-80b7-1b8840e76b6d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.126501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.127187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.127475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.127748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.128514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.128906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.129735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.130160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.130361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.130814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.132711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.134702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.141999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 802/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.143122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.154942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56edb3-c6c6-49b3-81b2-d164c065c408 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.160487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.164369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.164523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1589] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/users => generated 301 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:59.164642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.164642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.164833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.164932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.164932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.176390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa19542-fb50-4c17-bbfc-155f1fb1b49d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.178745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 803/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.179618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.185693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.186561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.186954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.187337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.188289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.190591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.194409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27dce-d04d-4dfb-973d-33919684c09a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.199595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.200258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.200535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.200815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.201549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.203241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.218872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 804/1591] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.220051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.233807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99beae1-0fe0-44d8-a4aa-f1acc25d5caa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.235349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1592] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/users => generated 289 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:30:59.236595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.242555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.243327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.243615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.243945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.244825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.247360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.251357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872e91-5d1e-4ec1-8738-0e0a67a5c915 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.257812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.258931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.259408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.259844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] PATH_INFO: `/v3/projects/c1a6059c78164a06ab14f53c41962199/users/df2c1f75eb414f869c54a6c2b1ab416d/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.260396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 805/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.261338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.275168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8aa03d-f2a5-4f97-93b8-c9fb704405b3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.279403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a6059c78164a06ab14f53c41962199, user_id=df2c1f75eb414f869c54a6c2b1ab416d, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.280281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.281562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.281562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.281778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] PATH_INFO: `/v3/projects/eb1df3e1bed34d18baa0dbf653bb2b8b/users/609f74b4a148429ba62059cb091c23d7/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.282831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.300285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1df3e1bed34d18baa0dbf653bb2b8b, user_id=609f74b4a148429ba62059cb091c23d7, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.304359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1594] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:30:59 2022] PUT /identity/v3/projects/c1a6059c78164a06ab14f53c41962199/users/df2c1f75eb414f869c54a6c2b1ab416d/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:30:59.304359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.304610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.321385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 806/1595] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:30:59 2022] PUT /identity/v3/projects/eb1df3e1bed34d18baa0dbf653bb2b8b/users/609f74b4a148429ba62059cb091c23d7/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:30:59.322348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.387718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:59.389198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1026ce-4de2-48f2-b4d6-7c50be012ccc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.406469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074393a-a674-4570-a607-6cbe10d69ed7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.413075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.413976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.414703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.414942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] PATH_INFO: `/v3/projects/12a3207b8cea465d8d1ed5c9e3963a4f/users/62766e7afd01401293df7e08707d6d95/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.433576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a3207b8cea465d8d1ed5c9e3963a4f, user_id=62766e7afd01401293df7e08707d6d95, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.436279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.451403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 807/1596] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:30:59 2022] PUT /identity/v3/projects/12a3207b8cea465d8d1ed5c9e3963a4f/users/62766e7afd01401293df7e08707d6d95/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:30:59.452453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.457028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.457709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.458377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.458377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.459121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.460791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.474435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.474435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.485479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a52d4-1d0c-4579-925f-7bb315eded6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.495477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:59.537500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9429105-e31a-4af0-bf1f-667affa7716b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.544139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.544770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.545054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.545330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.546059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.548478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.574670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 808/1598] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.574670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.582574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a22618-cf35-4a57-a45d-ab52f9ce6356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.583597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22618-cf35-4a57-a45d-ab52f9ce6356 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.584399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22618-cf35-4a57-a45d-ab52f9ce6356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.584856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a22618-cf35-4a57-a45d-ab52f9ce6356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.585812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22618-cf35-4a57-a45d-ab52f9ce6356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.586270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db240c-db90-4f37-a40f-2232ea30ad8b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.589245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a22618-cf35-4a57-a45d-ab52f9ce6356 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.590927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986cb19-156a-496a-b39f-374e79cd6003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.591821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986cb19-156a-496a-b39f-374e79cd6003 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.592092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986cb19-156a-496a-b39f-374e79cd6003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.592419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986cb19-156a-496a-b39f-374e79cd6003 admin admin] PATH_INFO: `/v3/projects/402f66e81fd14db9ac98506798ba4282/users/b1b1566726794259a515728a31a24a85/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.602832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.604988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958143e2-9631-43f4-abad-baa63e48088a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.605513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958143e2-9631-43f4-abad-baa63e48088a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.606005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958143e2-9631-43f4-abad-baa63e48088a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.615863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986cb19-156a-496a-b39f-374e79cd6003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=402f66e81fd14db9ac98506798ba4282, user_id=b1b1566726794259a515728a31a24a85, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.618615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986cb19-156a-496a-b39f-374e79cd6003 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.639034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-958143e2-9631-43f4-abad-baa63e48088a None None] MFA Rules not processed for user `df2c1f75eb414f869c54a6c2b1ab416d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:59.640276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-958143e2-9631-43f4-abad-baa63e48088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:59.640979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 809/1600] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:30:59 2022] PUT /identity/v3/projects/402f66e81fd14db9ac98506798ba4282/users/b1b1566726794259a515728a31a24a85/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:30:59.642087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.642396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.642706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.674982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] MFA Rules not processed for user `609f74b4a148429ba62059cb091c23d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:59.676133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:59.723894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1601] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:59.723894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.723894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.723894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.742104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:59.743658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:59.756539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 810/1602] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:59.756539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.791871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:59.816059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1603] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:59.817001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.823557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef0c4d-3220-4de4-8152-7ca2bf983259 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.826965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23def7-1905-4257-9c1f-c7b076ac9f3c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.828486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013cf02-7668-4dd2-9638-16fa4e86b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.829158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013cf02-7668-4dd2-9638-16fa4e86b55b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.829616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013cf02-7668-4dd2-9638-16fa4e86b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.829898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013cf02-7668-4dd2-9638-16fa4e86b55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.830689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013cf02-7668-4dd2-9638-16fa4e86b55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.831807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.832378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013cf02-7668-4dd2-9638-16fa4e86b55b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.832709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.832990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.833266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] PATH_INFO: `/v3/projects/c7916d019b0047f290705051769aabf9/users/4c7075c486364c90beceee1b8b571418/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.842466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 811/1604] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.843483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97745c-3388-48f6-b175-116d0525005f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.843794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97745c-3388-48f6-b175-116d0525005f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.844047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97745c-3388-48f6-b175-116d0525005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.848696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7916d019b0047f290705051769aabf9, user_id=4c7075c486364c90beceee1b8b571418, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.851332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.865440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1605] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:30:59 2022] PUT /identity/v3/projects/c7916d019b0047f290705051769aabf9/users/4c7075c486364c90beceee1b8b571418/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:30:59.866249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.875669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7f97745c-3388-48f6-b175-116d0525005f None None] MFA Rules not processed for user `4e42c2ed4d6b460c97cf9f8cf4747c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:59.877651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f97745c-3388-48f6-b175-116d0525005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:30:59.890984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:30:59.941382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533708b5-10f6-4f39-9680-7777d88c24f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:30:59.944604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 812/1606] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:30:59.945972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.946099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:30:59.946268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.946546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.946681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:30:59.946971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:30:59.947253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:30:59.952988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:30:59.954623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:30:59.965552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1607] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:30:59.966463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:30:59.975442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] MFA Rules not processed for user `b1b1566726794259a515728a31a24a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:30:59.976707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:00.026421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ff6b-e423-4a9c-86c1-3b97306b576a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.030253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.030903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.031326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.031610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.042377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 813/1608] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:00.043220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.055551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f0c92-5d4d-4e73-b7ba-04c215242d99 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.060307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.060984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.061275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.061595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.062321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.064381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.095045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 814/1609] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:00.095898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.104704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.106203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.116095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1610] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:00.116796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.125280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:00.130012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bedf89-5860-45b3-b90d-f01d913f5b24 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.158226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0c0fc-f527-486b-952f-1909a4ac3196 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.161977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.162631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.162923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.163211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.197828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e21514-fa3c-45b5-a66e-937302aa1ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.198665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e21514-fa3c-45b5-a66e-937302aa1ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.198949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e21514-fa3c-45b5-a66e-937302aa1ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.199186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e21514-fa3c-45b5-a66e-937302aa1ba2 admin admin] PATH_INFO: `/v3/projects/f43165885aec454c8f72eb44b5d8ba62/users/5f916c323dd942c1ac42c8dd6d42d552/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.208026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:00.216409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e21514-fa3c-45b5-a66e-937302aa1ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43165885aec454c8f72eb44b5d8ba62, user_id=5f916c323dd942c1ac42c8dd6d42d552, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.219352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e21514-fa3c-45b5-a66e-937302aa1ba2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.234370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.235281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1611] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:00 2022] PUT /identity/v3/projects/f43165885aec454c8f72eb44b5d8ba62/users/5f916c323dd942c1ac42c8dd6d42d552/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:00.236162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.241727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.242510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.243046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.249921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 815/1612] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:00.253789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.270863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52792594-37aa-47a9-be17-31874fc42618 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.277996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] MFA Rules not processed for user `4c7075c486364c90beceee1b8b571418`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:00.280001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:00.355229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2f3e9-35fc-40a8-a1db-b65a91498ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.356048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f3e9-35fc-40a8-a1db-b65a91498ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.356377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f3e9-35fc-40a8-a1db-b65a91498ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.356814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2f3e9-35fc-40a8-a1db-b65a91498ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.357618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1613] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:00.358815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.360643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2f3e9-35fc-40a8-a1db-b65a91498ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.364171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae2f3e9-35fc-40a8-a1db-b65a91498ab2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.374481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a9d8c-8226-4b5b-aa2d-85ff5d4958f8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.380520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4039f19-e022-4659-a9de-9cb32228e4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.381440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039f19-e022-4659-a9de-9cb32228e4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.382075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039f19-e022-4659-a9de-9cb32228e4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.382548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4039f19-e022-4659-a9de-9cb32228e4fb admin admin] PATH_INFO: `/v3/projects/2073f419769a42be967d132787f9ad1e/users/779f59f5eb434a758ee317f72da17fd5/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.401018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039f19-e022-4659-a9de-9cb32228e4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2073f419769a42be967d132787f9ad1e, user_id=779f59f5eb434a758ee317f72da17fd5, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.404410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4039f19-e022-4659-a9de-9cb32228e4fb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.405148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 816/1614] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:00.406857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ff022-dda8-41f5-9453-dff1fdb5f6e5 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.407251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ff022-dda8-41f5-9453-dff1fdb5f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.408053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ff022-dda8-41f5-9453-dff1fdb5f6e5 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.410119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 817/1615] 10.222.0.13 () {54 vars in 962 bytes} [Tue Jul 19 12:31:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:00.411088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ff022-dda8-41f5-9453-dff1fdb5f6e5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.424497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1616] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:00 2022] PUT /identity/v3/projects/2073f419769a42be967d132787f9ad1e/users/779f59f5eb434a758ee317f72da17fd5/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:00.427189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.427475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.427778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.460042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] MFA Rules not processed for user `5f916c323dd942c1ac42c8dd6d42d552`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:00.461275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:00.520425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ff022-dda8-41f5-9453-dff1fdb5f6e5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.525946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:00.562914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1617] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:00.562914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.594162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.594854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.595141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.595424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.596663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.598736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.629112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 818/1618] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:00.629881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.640742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:00.646026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ef02-f108-42a7-9bcb-d58439806773 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.651580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676400c8-b3e2-4124-bfbf-dfdcba8f0c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.652462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676400c8-b3e2-4124-bfbf-dfdcba8f0c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.652749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676400c8-b3e2-4124-bfbf-dfdcba8f0c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.653242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676400c8-b3e2-4124-bfbf-dfdcba8f0c3e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.653902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676400c8-b3e2-4124-bfbf-dfdcba8f0c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.656265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676400c8-b3e2-4124-bfbf-dfdcba8f0c3e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.665819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985a0f5-358a-4842-bc62-8d5f552e4c55 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.672454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28c058-4733-4055-95e5-4434016fca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.673500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28c058-4733-4055-95e5-4434016fca12 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.673893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28c058-4733-4055-95e5-4434016fca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.675387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28c058-4733-4055-95e5-4434016fca12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.676375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28c058-4733-4055-95e5-4434016fca12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.680285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28c058-4733-4055-95e5-4434016fca12 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.681855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 819/1619] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/users => generated 309 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:00.683108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.683496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.683829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.740320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:00.741820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:00.753959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1620] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:00.753959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.753959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.753959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.786402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:00.788254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:00.886375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1621] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:00.888134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.894718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 820/1622] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:00.895709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.903582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28425b95-9515-47ec-8adc-a5656ff7d8fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.910240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.911180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.911308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dcbec-7890-4242-8573-4d2dbd7d0a6a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.911800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.912488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.913182 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.916014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.916971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.917954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.918261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.918592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.919430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.921320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.932436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 821/1623] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:00.933335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.946129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed6ca0-7cae-4198-b54c-c4846034a43c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:00.950820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1624] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/projects => generated 401 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:00.951729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:00.954643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:00.956637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:00.957025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:00.957402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:00.958340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:00.960220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:00.979618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 822/1625] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:00.980588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.025251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:01.028855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2e229-3df5-40aa-9f9d-c70aa5fc6a13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.034895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.035657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.036204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.036570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.037382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.040090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.045131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5193f2-70f6-4007-a2ec-e16225453e4a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.065786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1626] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:00 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:01.066846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.108295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.109029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.109387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.109739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.111544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:01.121043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.122548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.140956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 823/1627] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:01.142015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.151761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568f6bb-807f-4a32-bc70-22040a3029e4 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.154186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a73b3-15fe-424f-bb13-9b08827eeb41 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.156599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.157907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.157907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.158241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] PATH_INFO: `/v3/projects/442f5b696a4a4997889b795573e48b7c/users/95ed66bab9fb4eebb5af01cfd098f4ba/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.163877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.164641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.165480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.165480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.184110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=442f5b696a4a4997889b795573e48b7c, user_id=95ed66bab9fb4eebb5af01cfd098f4ba, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.187891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.223897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 824/1628] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:01 2022] PUT /identity/v3/projects/442f5b696a4a4997889b795573e48b7c/users/95ed66bab9fb4eebb5af01cfd098f4ba/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:01.227818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.293342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.298011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.327589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a20ed5-7fb8-4ba4-a0b8-5fd850dbbbfa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.333857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.334002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1629] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:01.335006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.335425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.335866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.336365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.337590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.339878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.375326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 825/1630] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:01 2022] POST /identity/v3/users => generated 307 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:01.376310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.376842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850bf4-cb9e-4b0e-ad76-272ea8181361 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.390715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1723-2438-4be7-a012-e01feef7bfdf admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.391687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.394215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.394681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.395040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] PATH_INFO: `/v3/projects/44bc6b865ae042fe93b2d83f742d0300/users/76c9e0cd0e5640088c63d0c7c4fe8307/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.397715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.398638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.399501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.399893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.400826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.402821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.420911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 826/1631] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:01.421976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.430885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bc6b865ae042fe93b2d83f742d0300, user_id=76c9e0cd0e5640088c63d0c7c4fe8307, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.433749 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.435069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163e165-4f0f-42cf-b1b6-37d213c70260 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.446432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.447520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.447842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.448206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.449004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.450627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.453997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1632] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:01 2022] PUT /identity/v3/projects/44bc6b865ae042fe93b2d83f742d0300/users/76c9e0cd0e5640088c63d0c7c4fe8307/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:01.454877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.480052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 827/1633] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:01.480798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.506457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbcdd0-66ea-4c7d-9501-3441d9be1d5e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.542452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361a62-85b9-47a5-b415-2b1465ff0581 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.548553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.550225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.550537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.550854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.552522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.554379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.568353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1634] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:01.569594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.572484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:01.586638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2cf15-f789-4e6e-9578-6f9c4548e159 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.593406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.593406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.593406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.593855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.594525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.597153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.612090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1635] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:01.613364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.626899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.628627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20334021-c4fa-4676-9dc9-6e7c0da40d6a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.629008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.629008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.629165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] PATH_INFO: `/v3/projects/febf027f55a64fd1a3db1c375278c1b2/users/5dc0a15f27ee4b9c9ba9d758eaad02bb/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.635461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.637333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.637333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.637655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] PATH_INFO: `/v3/projects/442f5b696a4a4997889b795573e48b7c/users/95ed66bab9fb4eebb5af01cfd098f4ba/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.640475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=442f5b696a4a4997889b795573e48b7c, user_id=95ed66bab9fb4eebb5af01cfd098f4ba, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.643776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.665617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febf027f55a64fd1a3db1c375278c1b2, user_id=5dc0a15f27ee4b9c9ba9d758eaad02bb, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.666780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1636] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:01 2022] PUT /identity/v3/projects/442f5b696a4a4997889b795573e48b7c/users/95ed66bab9fb4eebb5af01cfd098f4ba/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:01.667740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.669603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.695823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:01.702271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 828/1637] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:01 2022] PUT /identity/v3/projects/febf027f55a64fd1a3db1c375278c1b2/users/5dc0a15f27ee4b9c9ba9d758eaad02bb/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:01.704261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.755865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbad94-77a8-4bf1-b512-3a8c40a21ee6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.814907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b029c-6d72-4fd6-a2e2-f70ebebe2296 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:01.820042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfa2d5-d187-48f1-8c2c-9f504ffd256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.821355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa2d5-d187-48f1-8c2c-9f504ffd256b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.821733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa2d5-d187-48f1-8c2c-9f504ffd256b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.822013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfa2d5-d187-48f1-8c2c-9f504ffd256b admin admin] PATH_INFO: `/v3/projects/44bc6b865ae042fe93b2d83f742d0300/users/76c9e0cd0e5640088c63d0c7c4fe8307/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.823234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:01.824171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfa2d5-d187-48f1-8c2c-9f504ffd256b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bc6b865ae042fe93b2d83f742d0300, user_id=76c9e0cd0e5640088c63d0c7c4fe8307, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.826148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.826548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.827036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] PATH_INFO: `/v3/projects/718bcf6ce0b54c2f915e76bdf6a67a72/users/62d7825cd59e4987ae313da1ba27bc1d/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.827168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfa2d5-d187-48f1-8c2c-9f504ffd256b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.845316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 829/1638] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:01 2022] PUT /identity/v3/projects/44bc6b865ae042fe93b2d83f742d0300/users/76c9e0cd0e5640088c63d0c7c4fe8307/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:01.846817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:01.847161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:01.847450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:01.849295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718bcf6ce0b54c2f915e76bdf6a67a72, user_id=62d7825cd59e4987ae313da1ba27bc1d, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:01.852752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:01.882940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1639] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:01 2022] PUT /identity/v3/projects/718bcf6ce0b54c2f915e76bdf6a67a72/users/62d7825cd59e4987ae313da1ba27bc1d/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:01.884807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] MFA Rules not processed for user `95ed66bab9fb4eebb5af01cfd098f4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:01.886393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:01.886657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:01.982753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 830/1640] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:01.984011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:02.054269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93888cb0-889f-457d-bc2c-2985c3da1d74 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.073195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:02.075025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebc33e-8242-4772-9bb4-018352fd9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.075909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebc33e-8242-4772-9bb4-018352fd9b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.076420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebc33e-8242-4772-9bb4-018352fd9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.076890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebc33e-8242-4772-9bb4-018352fd9b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.077859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebc33e-8242-4772-9bb4-018352fd9b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.079826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebc33e-8242-4772-9bb4-018352fd9b50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.120641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1641] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:02.124506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f326d2-0747-4571-809a-724739c4c6fd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.124783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f326d2-0747-4571-809a-724739c4c6fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.125096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f326d2-0747-4571-809a-724739c4c6fd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.130766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1642] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jul 19 12:31:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:02.131540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f043400-a008-49dd-a03c-0bd2e9fffadd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.131694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f326d2-0747-4571-809a-724739c4c6fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:02.144730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.146085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.147653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.148405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.150373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.154101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.182305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 831/1643] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:02.184725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:02.211257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadcd58-e7ba-48bf-b6d9-8eb795b403a9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.224779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b0917-4b98-4c65-ab53-97b3c38f8fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.226484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0917-4b98-4c65-ab53-97b3c38f8fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.228086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0917-4b98-4c65-ab53-97b3c38f8fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.229372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b0917-4b98-4c65-ab53-97b3c38f8fc0 admin admin] PATH_INFO: `/v3/projects/febf027f55a64fd1a3db1c375278c1b2/users/5dc0a15f27ee4b9c9ba9d758eaad02bb/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.233379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:02.234551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b0917-4b98-4c65-ab53-97b3c38f8fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febf027f55a64fd1a3db1c375278c1b2, user_id=5dc0a15f27ee4b9c9ba9d758eaad02bb, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.243555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b0917-4b98-4c65-ab53-97b3c38f8fc0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.247832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f326d2-0747-4571-809a-724739c4c6fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.255903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.256685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.256979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.257322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.272608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 832/1644] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:02 2022] PUT /identity/v3/projects/febf027f55a64fd1a3db1c375278c1b2/users/5dc0a15f27ee4b9c9ba9d758eaad02bb/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:02.275728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81758694-95b4-46f0-bb60-327c8b584da6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.277244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81758694-95b4-46f0-bb60-327c8b584da6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.278201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81758694-95b4-46f0-bb60-327c8b584da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.287320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.288909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.306919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-81758694-95b4-46f0-bb60-327c8b584da6 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:02.308403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-81758694-95b4-46f0-bb60-327c8b584da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:02.315756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:02.316646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:02.338534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f2818-9b8f-4d24-8f17-2c46696d604b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.438677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 833/1646] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:02.441094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a3fcd-c3e2-4cc8-8924-98b0003163c8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.441485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a3fcd-c3e2-4cc8-8924-98b0003163c8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.441954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a3fcd-c3e2-4cc8-8924-98b0003163c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.466982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.467892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.468201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.468483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] PATH_INFO: `/v3/projects/718bcf6ce0b54c2f915e76bdf6a67a72/users/62d7825cd59e4987ae313da1ba27bc1d/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.473416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718bcf6ce0b54c2f915e76bdf6a67a72, user_id=62d7825cd59e4987ae313da1ba27bc1d, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.475522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.491310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8e9a3fcd-c3e2-4cc8-8924-98b0003163c8 None None] MFA Rules not processed for user `5dc0a15f27ee4b9c9ba9d758eaad02bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:02.493311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8e9a3fcd-c3e2-4cc8-8924-98b0003163c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:02.513374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1647] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:02 2022] PUT /identity/v3/projects/718bcf6ce0b54c2f915e76bdf6a67a72/users/62d7825cd59e4987ae313da1ba27bc1d/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:02.522990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:02.594276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e9604-9ce2-4bdf-b576-10d2f48792c1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.600070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.601275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.601743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.602187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.657538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 834/1648] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:02.659147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b52725-bef4-44e9-8494-37fd87ef234c None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.659441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b52725-bef4-44e9-8494-37fd87ef234c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.659727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b52725-bef4-44e9-8494-37fd87ef234c None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.669050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 835/1649] 10.222.0.13 () {54 vars in 962 bytes} [Tue Jul 19 12:31:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:02.670406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82a63f-a6f0-44f6-b2b3-fc2a3c914f01 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.671565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82a63f-a6f0-44f6-b2b3-fc2a3c914f01 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.671565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82a63f-a6f0-44f6-b2b3-fc2a3c914f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.702434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bb82a63f-a6f0-44f6-b2b3-fc2a3c914f01 None None] MFA Rules not processed for user `62d7825cd59e4987ae313da1ba27bc1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:02.703954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb82a63f-a6f0-44f6-b2b3-fc2a3c914f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:02.723212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.725676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.742054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1650] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:02.743223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:02.747484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:02.758236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b15f8-f517-4d13-a1e3-5e15ccb7b010 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:02.764667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:02.766089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.766382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.766741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.797478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:02.827313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 836/1651] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:02.830118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb91bde-c1fa-464d-a640-9b4eefec4d28 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.830852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb91bde-c1fa-464d-a640-9b4eefec4d28 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.831417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb91bde-c1fa-464d-a640-9b4eefec4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.854762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0bb91bde-c1fa-464d-a640-9b4eefec4d28 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:02.858782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0bb91bde-c1fa-464d-a640-9b4eefec4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:02.878033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:02.879951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:02.880656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 837/1652] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Jul 19 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:02.882963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafe1a-8504-4955-807f-4f6bbf967768 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.883481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafe1a-8504-4955-807f-4f6bbf967768 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.884014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feafe1a-8504-4955-807f-4f6bbf967768 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.891999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 838/1653] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jul 19 12:31:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:02.893130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1654] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:02.894174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:02.894525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:02.894874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:02.915329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:02.916615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:02.940522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 839/1655] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:03.161034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:03.180225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c8758-a198-4336-b510-b63aac709912 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:03.190838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:03.196082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:03.197311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:03.197628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:03.244900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:03.247083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:03.276981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1656] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:03.357830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:03.372441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b3756d-ab47-46f4-81c7-0c885861c06e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:03.382793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a0994-6e0a-4d6a-b1d2-3c2d255bad78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:03.383499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0994-6e0a-4d6a-b1d2-3c2d255bad78 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:03.383789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0994-6e0a-4d6a-b1d2-3c2d255bad78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:03.384206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0994-6e0a-4d6a-b1d2-3c2d255bad78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:03.406285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a0994-6e0a-4d6a-b1d2-3c2d255bad78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:03.407776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a0994-6e0a-4d6a-b1d2-3c2d255bad78 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:03.441527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 840/1657] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:03.489171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:03.572562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288675d2-e8ea-4785-a4fc-c974ac27bdcf service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:03.572562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:03.572562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:03.572562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:03.573777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:03.573777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:03.573777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:03.575605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1658] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:04.182304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:04.182856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:04.182856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:04.243157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] MFA Rules not processed for user `76c9e0cd0e5640088c63d0c7c4fe8307`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:04.245634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:04.477155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 841/1659] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:04.543418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:04.569636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9240abb-4f4f-43a7-9798-5f2bba0afe7c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:04.586427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca14ac-b70e-4848-841a-d120369301ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:04.588866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca14ac-b70e-4848-841a-d120369301ea service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:04.591104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca14ac-b70e-4848-841a-d120369301ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:04.591872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca14ac-b70e-4848-841a-d120369301ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:04.802671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca14ac-b70e-4848-841a-d120369301ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:04.814444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca14ac-b70e-4848-841a-d120369301ea service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:04.880120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1660] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:06.659505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:06.664682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a57f-a399-41a5-b473-f40498b24d70 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:06.665201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a57f-a399-41a5-b473-f40498b24d70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:06.665270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a57f-a399-41a5-b473-f40498b24d70 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:06.666972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1661] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 19 12:31:06.697383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368bda7-117e-4077-b70a-b747bdeeed46 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:06.697677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368bda7-117e-4077-b70a-b747bdeeed46 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:06.698376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368bda7-117e-4077-b70a-b747bdeeed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:06.733735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b368bda7-117e-4077-b70a-b747bdeeed46 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:06.739380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b368bda7-117e-4077-b70a-b747bdeeed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:06.800750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc29bd-792e-4b86-81f6-ecd2ee347a84 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:06.812407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:06.813513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:06.814062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:06.815372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] PATH_INFO: `/v3/users/4c7075c486364c90beceee1b8b571418` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:06.819364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7075c486364c90beceee1b8b571418)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:06.826022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:06.897849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:06.901803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1662] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 19 12:31:06.918767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020694fd-96c6-452c-a273-e76ac2a70493 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:06.919190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020694fd-96c6-452c-a273-e76ac2a70493 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:06.919579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020694fd-96c6-452c-a273-e76ac2a70493 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:06.922079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1663] 10.222.0.13 () {54 vars in 961 bytes} [Tue Jul 19 12:31:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:06.929930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:06.930370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:06.930864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:06.957622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7075c486364c90beceee1b8b571418'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:06.960824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:06.962794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:06.978943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1664] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Jul 19 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:06.981035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7075c486364c90beceee1b8b571418'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:06.982111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7075c486364c90beceee1b8b571418'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:06.990162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:07.008140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3daf1-9e1c-4ae9-be3b-25b333a01d79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:07.009885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7075c486364c90beceee1b8b571418'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:07.025578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:07.026300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c8e20a9-b3cd-41af-91bf-74d6693e7784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7075c486364c90beceee1b8b571418'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:07.027752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:07.028762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:07.030216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:07.035756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:07.045724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 842/1665] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:06 2022] DELETE /identity/v3/users/4c7075c486364c90beceee1b8b571418 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:07.050665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:07.054107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:07.055878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:07.068912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:07.074463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:07.101796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1666] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:07.106275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:07.107594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:07.215256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 843/1667] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:07.275466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:07.277174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:07.306725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:07.448393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70462-57c2-45d8-a8cc-3b7d010fce89 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:07.530574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:07.531190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:07.531608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:07.532294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:07.544327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:07.545959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:07.558642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1668] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:08.245823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:08.247961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:08.268259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04800ea5-3b03-4f8f-87d0-d192f45e7ed8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:08.280418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:08.281321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:08.281603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:08.281879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] PATH_INFO: `/v3/users/4e42c2ed4d6b460c97cf9f8cf4747c9a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:08.283211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e42c2ed4d6b460c97cf9f8cf4747c9a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:08.285283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:08.354175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e42c2ed4d6b460c97cf9f8cf4747c9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.375872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e42c2ed4d6b460c97cf9f8cf4747c9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.377127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e42c2ed4d6b460c97cf9f8cf4747c9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.396887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e42c2ed4d6b460c97cf9f8cf4747c9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.407119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e42c2ed4d6b460c97cf9f8cf4747c9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.415112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 844/1669] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:08 2022] DELETE /identity/v3/users/4e42c2ed4d6b460c97cf9f8cf4747c9a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:08.725284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:08.741845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026b761-2bc5-4d26-8c97-09b70aa9624a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:08.821117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:08.823300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:08.823672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:08.824206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] PATH_INFO: `/v3/projects/74d80918a03945e6adceec468320d033` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:08.826100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d80918a03945e6adceec468320d033)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:08.828778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:08.898017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d80918a03945e6adceec468320d033', 'request_id': 'req-0556d700-5d3e-4399-882a-c98fa1d6f5a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.916335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d80918a03945e6adceec468320d033', 'request_id': 'req-0556d700-5d3e-4399-882a-c98fa1d6f5a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.927622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d80918a03945e6adceec468320d033', 'request_id': 'req-0556d700-5d3e-4399-882a-c98fa1d6f5a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.937909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:08.944578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] The token cache is being invalidate because project 74d80918a03945e6adceec468320d033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:08.945323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.946380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:08.948779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1670] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:08 2022] DELETE /identity/v3/projects/74d80918a03945e6adceec468320d033 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:08.952792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:08.955142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:08.981620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6ade8-924d-4f0d-a34c-391f084f4174 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:08.996884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:09.087950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:09.089820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:09.089914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:09.090351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:09.091272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:09.091515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] PATH_INFO: `/v3/users/76c9e0cd0e5640088c63d0c7c4fe8307` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:09.094179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c9e0cd0e5640088c63d0c7c4fe8307)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:09.097909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:09.115791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556d700-5d3e-4399-882a-c98fa1d6f5a0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:09.121803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:09.126110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:09.126805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:09.127352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] PATH_INFO: `/v3/projects/c7916d019b0047f290705051769aabf9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:09.129756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7916d019b0047f290705051769aabf9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:09.132543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:09.199567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c9e0cd0e5640088c63d0c7c4fe8307'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.214310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c9e0cd0e5640088c63d0c7c4fe8307'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.214962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c9e0cd0e5640088c63d0c7c4fe8307'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.240669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c9e0cd0e5640088c63d0c7c4fe8307'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.253494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7916d019b0047f290705051769aabf9', 'request_id': 'req-1fcc912f-d024-4988-8de5-2edf59ba4dbd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.255783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c9e0cd0e5640088c63d0c7c4fe8307'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.267232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7916d019b0047f290705051769aabf9', 'request_id': 'req-1fcc912f-d024-4988-8de5-2edf59ba4dbd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.273629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 845/1671] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:08 2022] DELETE /identity/v3/users/76c9e0cd0e5640088c63d0c7c4fe8307 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:09.293075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7916d019b0047f290705051769aabf9', 'request_id': 'req-1fcc912f-d024-4988-8de5-2edf59ba4dbd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.325712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] The token cache is being invalidate because project c7916d019b0047f290705051769aabf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:09.326716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.327711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.332892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1672] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:09 2022] DELETE /identity/v3/projects/c7916d019b0047f290705051769aabf9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:09.337147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:09.338827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:09.437962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f70870-d10d-44b8-b061-e573184ad589 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:09.446438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:09.469688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:09.469982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:09.470269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] PATH_INFO: `/v3/users/5dc0a15f27ee4b9c9ba9d758eaad02bb` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:09.475186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc0a15f27ee4b9c9ba9d758eaad02bb)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:09.477200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:09.570754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:09.572506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:09.580667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:09.592321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc0a15f27ee4b9c9ba9d758eaad02bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.610021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc0a15f27ee4b9c9ba9d758eaad02bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.611006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc0a15f27ee4b9c9ba9d758eaad02bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.633157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc0a15f27ee4b9c9ba9d758eaad02bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.641458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a900c498-0745-47bd-afd8-2432c1049b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc0a15f27ee4b9c9ba9d758eaad02bb'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:09.664287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 846/1673] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:09 2022] DELETE /identity/v3/users/5dc0a15f27ee4b9c9ba9d758eaad02bb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:09.734846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc912f-d024-4988-8de5-2edf59ba4dbd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:09.882399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:09.883904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:09.884299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:09.885307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] PATH_INFO: `/v3/projects/44bc6b865ae042fe93b2d83f742d0300` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:09.886401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bc6b865ae042fe93b2d83f742d0300)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:09.888740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:09.976055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:09.997992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bc6b865ae042fe93b2d83f742d0300', 'request_id': 'req-1c7ebc30-476e-4133-87be-3a166441f3c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.005759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bc6b865ae042fe93b2d83f742d0300', 'request_id': 'req-1c7ebc30-476e-4133-87be-3a166441f3c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.015942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bc6b865ae042fe93b2d83f742d0300', 'request_id': 'req-1c7ebc30-476e-4133-87be-3a166441f3c3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.032448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] The token cache is being invalidate because project 44bc6b865ae042fe93b2d83f742d0300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:10.032744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.034369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1c7ebc30-476e-4133-87be-3a166441f3c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.039427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1674] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:09 2022] DELETE /identity/v3/projects/44bc6b865ae042fe93b2d83f742d0300 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:10.044688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.045333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.045934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.071269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:10.072639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:10.180215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 847/1675] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:10.187916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.188414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.188833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.214267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:10.215903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:10.235955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1676] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:10.240491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.254638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17071ea5-a0e4-4280-9efb-a9b7f6a1ee16 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.271838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.273373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.273928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.274445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.275732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:10.276235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.279409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.296144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 848/1677] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:10.301233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.317360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33174323-c559-4386-89c2-4c6373ae0796 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.328266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.329308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.330285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.331284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.333358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.336909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.377655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1678] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:10.382285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.395572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee4060-39bb-4113-837f-d194251b6239 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.470096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.477020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:10.482500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.483268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.483666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.484023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.484836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.486987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.496362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80200747-d2ef-4dde-8f0f-34f4600d9a45 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.506589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.507268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.507565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.508019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] PATH_INFO: `/v3/projects/febf027f55a64fd1a3db1c375278c1b2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.518537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] RBAC: Authorizing `identity:delete_project(project_id=febf027f55a64fd1a3db1c375278c1b2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.522384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.527910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 849/1679] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/users => generated 282 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:10.529128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.540328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9653d4da-1506-42a6-a385-1939221b3a86 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.547735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.549248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.549554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.549842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.551102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.554927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.571476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 850/1680] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:10.575429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.593245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4513-4a87-4f48-9001-18c7f9f33f83 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.606017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.606889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.607184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.607464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.608268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.612319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.634963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febf027f55a64fd1a3db1c375278c1b2', 'request_id': 'req-0d30c92c-22cb-49f0-be84-7798a978748d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.643669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febf027f55a64fd1a3db1c375278c1b2', 'request_id': 'req-0d30c92c-22cb-49f0-be84-7798a978748d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.658652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febf027f55a64fd1a3db1c375278c1b2', 'request_id': 'req-0d30c92c-22cb-49f0-be84-7798a978748d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.664643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 851/1681] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:10.666861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:10.680647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] The token cache is being invalidate because project febf027f55a64fd1a3db1c375278c1b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:10.681096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.681850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d30c92c-22cb-49f0-be84-7798a978748d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:10.684753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1682] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:10 2022] DELETE /identity/v3/projects/febf027f55a64fd1a3db1c375278c1b2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:10.685106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da076c93-22c9-407d-9c6b-fc05424a0aef admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:10.686458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a76b5-1fae-4ca5-a457-67ad0e39509a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.686808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a76b5-1fae-4ca5-a457-67ad0e39509a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.687150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a76b5-1fae-4ca5-a457-67ad0e39509a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.698285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:10.749224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa7a76b5-1fae-4ca5-a457-67ad0e39509a None None] MFA Rules not processed for user `62766e7afd01401293df7e08707d6d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:10.750558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa7a76b5-1fae-4ca5-a457-67ad0e39509a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:10.861295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73ace3-a284-4bc5-9c42-6c65fe0e990c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:10.862210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ace3-a284-4bc5-9c42-6c65fe0e990c admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.862497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ace3-a284-4bc5-9c42-6c65fe0e990c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.862783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73ace3-a284-4bc5-9c42-6c65fe0e990c admin admin] PATH_INFO: `/v3/projects/a9106d1dd4b3413ab60d170187627d54/users/8a31429b757c4d8baf4d8fbf62218872/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.892793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1683] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:10.898603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.898914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.899200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.901100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ace3-a284-4bc5-9c42-6c65fe0e990c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9106d1dd4b3413ab60d170187627d54, user_id=8a31429b757c4d8baf4d8fbf62218872, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:10.905818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73ace3-a284-4bc5-9c42-6c65fe0e990c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:10.912098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:10.944025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 852/1684] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:10 2022] PUT /identity/v3/projects/a9106d1dd4b3413ab60d170187627d54/users/8a31429b757c4d8baf4d8fbf62218872/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:10.944025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db013b15-06f7-486d-9abc-641be7123179 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:10.944025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db013b15-06f7-486d-9abc-641be7123179 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:10.944025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db013b15-06f7-486d-9abc-641be7123179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:10.944458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:10.944458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:10.979214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-db013b15-06f7-486d-9abc-641be7123179 None None] MFA Rules not processed for user `62766e7afd01401293df7e08707d6d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:10.981815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db013b15-06f7-486d-9abc-641be7123179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:11.070018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 853/1685] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:11.070018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83ca83-9d66-4c80-9c91-b3193df5c46f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.070018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83ca83-9d66-4c80-9c91-b3193df5c46f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.070018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83ca83-9d66-4c80-9c91-b3193df5c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.110275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1686] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:11.111341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.114848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-cf83ca83-9d66-4c80-9c91-b3193df5c46f None None] MFA Rules not processed for user `8a31429b757c4d8baf4d8fbf62218872`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:11.116238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:11.116376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cf83ca83-9d66-4c80-9c91-b3193df5c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:11.200886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 854/1687] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:11.202284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.202583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.202864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.228024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:11.229364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:11.234458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053dca5-8e56-4201-a99c-0ae7a220ff6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.241313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.242036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.242323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.242607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.243995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 855/1688] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:11.245316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.247162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:11.260659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2315d-90e9-4118-be37-d705a5d6a4b4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.265947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.266628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.266915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.267196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.268497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.268688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.270306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.270778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.291816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1689] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:11.292610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.311284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d0374-24dc-4ea1-ab74-ea77b48e7515 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.314555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:11.322544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.323474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.325059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.325494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.329416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:11.329866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.333424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.333991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 856/1690] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:11 2022] POST /identity/v3/projects => generated 363 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:11.334868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.352506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1691] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:11.357875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.372423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd78cc2-bc49-473b-bfba-a463449805f6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.527729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ad7a8-3bd8-4413-b17c-8743ee76e4f4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.529067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.529804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.530098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.530378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.531084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.541064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.552254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.552906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.553196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.553494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.554205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.556260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.618088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 857/1692] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:11 2022] POST /identity/v3/users => generated 288 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:11.621015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.627777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1693] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:11 2022] POST /identity/v3/projects => generated 355 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:11.631737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.640308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:11.642255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9a02a-f128-4b31-9aae-95ad3335caf7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.752304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b254-fe23-4fa4-8439-67db976567e6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.771072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.771766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.772053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.773525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.775793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.777877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.834505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1694] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:11 2022] POST /identity/v3/users => generated 278 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:11.838483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.850182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.853054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.853543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.855012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.856345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.862657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.872455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a00fe-82f2-4327-8fcd-0604680ceb00 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.896875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 858/1695] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:11.898739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.904079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.904837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.905129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.905421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.906175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.907988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.928513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8a8ac-11d1-4071-a244-de944f716594 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.938152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1696] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:11.942197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.943480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.945084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.946479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.946882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] PATH_INFO: `/v3/projects/2e45b2129b684e6899457aa3b3e4dd90/users/0781a9fc92a64a2088b00c1e1a525903/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.957436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f1e84-f53f-4489-98c6-f3b92c6d3bd1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:11.967359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:11.968572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:11.969333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:11.969798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:11.971471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:11.973536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:11.988920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1697] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:11.992835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:11.999652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e45b2129b684e6899457aa3b3e4dd90, user_id=0781a9fc92a64a2088b00c1e1a525903, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:12.007228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:12.008088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232863-8b94-4899-aafc-ea9943fbff4f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:12.014081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a57a93-39be-49e8-a7c7-e877656bfe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:12.015901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a57a93-39be-49e8-a7c7-e877656bfe05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.016550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a57a93-39be-49e8-a7c7-e877656bfe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.016991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a57a93-39be-49e8-a7c7-e877656bfe05 admin admin] PATH_INFO: `/v3/projects/219ebde4c8294d928da98b603d4ecb05/users/1e41fa711df7482a8c2d682eeb1557b2/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.039143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a57a93-39be-49e8-a7c7-e877656bfe05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=219ebde4c8294d928da98b603d4ecb05, user_id=1e41fa711df7482a8c2d682eeb1557b2, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:12.043962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a57a93-39be-49e8-a7c7-e877656bfe05 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:12.049236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 859/1698] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:11 2022] PUT /identity/v3/projects/2e45b2129b684e6899457aa3b3e4dd90/users/0781a9fc92a64a2088b00c1e1a525903/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:12.051587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:12.062856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1699] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:11 2022] PUT /identity/v3/projects/219ebde4c8294d928da98b603d4ecb05/users/1e41fa711df7482a8c2d682eeb1557b2/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:12.074454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.074981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.075402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.146814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] MFA Rules not processed for user `1e41fa711df7482a8c2d682eeb1557b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:12.148741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:12.226652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf58d90-a055-46fc-b34f-58b92a5dbde3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:12.231846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:12.276327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1700] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:12.277508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:12.394072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a80e11-78d9-4d72-8a51-53a03193701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:12.403937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a80e11-78d9-4d72-8a51-53a03193701c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.404429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a80e11-78d9-4d72-8a51-53a03193701c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.406599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:12.407263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a80e11-78d9-4d72-8a51-53a03193701c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.409386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a80e11-78d9-4d72-8a51-53a03193701c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:12.414200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a80e11-78d9-4d72-8a51-53a03193701c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:12.453728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91da30-857c-4aee-98b2-3f250f2892e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:12.458797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f486cd-97e2-41ba-a7fc-f081eb0ba489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:12.463787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f486cd-97e2-41ba-a7fc-f081eb0ba489 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.464304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f486cd-97e2-41ba-a7fc-f081eb0ba489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.464716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f486cd-97e2-41ba-a7fc-f081eb0ba489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.465592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f486cd-97e2-41ba-a7fc-f081eb0ba489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:12.468049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f486cd-97e2-41ba-a7fc-f081eb0ba489 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:12.478778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 860/1701] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:12.480644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12408d78-9473-4a6a-91dd-895759b37ad4 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.485326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12408d78-9473-4a6a-91dd-895759b37ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.485718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12408d78-9473-4a6a-91dd-895759b37ad4 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.493540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 861/1702] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jul 19 12:31:12 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:12.496598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12408d78-9473-4a6a-91dd-895759b37ad4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:12.540374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12408d78-9473-4a6a-91dd-895759b37ad4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:12.541497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1703] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:12 2022] POST /identity/v3/projects => generated 357 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:12.543251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.543549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.544275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.667434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:12.669427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:12.692044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede5128-ab16-4971-b480-1413fecd1e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:12.693254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede5128-ab16-4971-b480-1413fecd1e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.693776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede5128-ab16-4971-b480-1413fecd1e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.694472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede5128-ab16-4971-b480-1413fecd1e43 admin admin] PATH_INFO: `/v3/projects/2e45b2129b684e6899457aa3b3e4dd90/users/0781a9fc92a64a2088b00c1e1a525903/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.705744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede5128-ab16-4971-b480-1413fecd1e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e45b2129b684e6899457aa3b3e4dd90, user_id=0781a9fc92a64a2088b00c1e1a525903, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:12.712558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede5128-ab16-4971-b480-1413fecd1e43 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:12.746672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 862/1704] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:12 2022] PUT /identity/v3/projects/2e45b2129b684e6899457aa3b3e4dd90/users/0781a9fc92a64a2088b00c1e1a525903/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:12.749568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87285965-fb5e-4614-b102-71fc6c63adeb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:12.751290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87285965-fb5e-4614-b102-71fc6c63adeb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:12.751733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87285965-fb5e-4614-b102-71fc6c63adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:12.841780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-87285965-fb5e-4614-b102-71fc6c63adeb None None] MFA Rules not processed for user `779f59f5eb434a758ee317f72da17fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:12.844959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-87285965-fb5e-4614-b102-71fc6c63adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:12.871645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1705] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:12.872929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:12.909280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a56f92-a338-441a-81ce-8d759e359c58 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.032583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 863/1706] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:13.034027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18ab78-0a10-44dd-9276-2a7dfdf61f90 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.034314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18ab78-0a10-44dd-9276-2a7dfdf61f90 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.034591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18ab78-0a10-44dd-9276-2a7dfdf61f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.100947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ec18ab78-0a10-44dd-9276-2a7dfdf61f90 None None] MFA Rules not processed for user `0781a9fc92a64a2088b00c1e1a525903`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:13.102237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ec18ab78-0a10-44dd-9276-2a7dfdf61f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:13.106508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.108852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.110017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.110395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.112940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.118967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.130238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:13.164105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1707] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:12 2022] POST /identity/v3/users => generated 285 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:13.164105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:13.166110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:13.280807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 864/1708] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:13.282561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.283088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.283555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.322148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] MFA Rules not processed for user `779f59f5eb434a758ee317f72da17fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:13.323997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:13.351210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:13.352911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 865/1709] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:13.354938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:13.364288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55ca27-062e-4ff4-85e2-d1d154fca57f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.374542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.384049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.384201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9576d1e-e89f-4c93-92c7-929d693fdeb9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.384993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.386984 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.398620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.401324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.401786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.402085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.404692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.406406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.407310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.413726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.439036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1710] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:13.440325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 866/1711] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:13.447793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:13.477128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4379c25-c471-4911-857a-ee83bea3bd23 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.497840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.499608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.499825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.500013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] PATH_INFO: `/v3/projects/5d9beebcdc3b4e08ad90eb55adde32ba/users/57d018da8d8f49bb8216bd3340a6d9d5/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.543202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:13.548238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:13.552110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9beebcdc3b4e08ad90eb55adde32ba, user_id=57d018da8d8f49bb8216bd3340a6d9d5, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.555877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.576046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1712] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:13 2022] PUT /identity/v3/projects/5d9beebcdc3b4e08ad90eb55adde32ba/users/57d018da8d8f49bb8216bd3340a6d9d5/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:13.579508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:13.679136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceb9a5-7e05-4246-af9e-ec271a1dd311 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.684207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.685013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.685394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.685777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.686647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.688581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.704211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1713] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:13.707650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:13.729478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36377c-9f01-431a-8061-4458a47d2715 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.743444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c374275-470e-469d-9b2b-1306798a6853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.744831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374275-470e-469d-9b2b-1306798a6853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.745122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374275-470e-469d-9b2b-1306798a6853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.745407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c374275-470e-469d-9b2b-1306798a6853 admin admin] PATH_INFO: `/v3/projects/5d9beebcdc3b4e08ad90eb55adde32ba/users/57d018da8d8f49bb8216bd3340a6d9d5/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.747609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c374275-470e-469d-9b2b-1306798a6853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9beebcdc3b4e08ad90eb55adde32ba, user_id=57d018da8d8f49bb8216bd3340a6d9d5, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.752564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c374275-470e-469d-9b2b-1306798a6853 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.768376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1714] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:13 2022] PUT /identity/v3/projects/5d9beebcdc3b4e08ad90eb55adde32ba/users/57d018da8d8f49bb8216bd3340a6d9d5/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:13.774408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ac84e-47ab-487f-9585-56341f560c35 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:13.777419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.777890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.778309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.813576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] MFA Rules not processed for user `57d018da8d8f49bb8216bd3340a6d9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:13.816721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:13.881960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:13.882811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:13.883096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:13.883376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:13.974259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:13.975728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:13.988682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 867/1715] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:13.989709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1716] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:13.994050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:14.070648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:14.081574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d1794-1de1-4979-9f7b-09442655d13b service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:14.092868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:14.094238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:14.094734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:14.095411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:14.109292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:14.110751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:14.123216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 868/1717] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:14.131952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:14.133541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:14.227645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:14.327829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110efb9-aca9-4b9c-8e0b-29c5dc0381dd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:14.346667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:14.347342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:14.347620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:14.347901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:14.356016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:14.357434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:14.376593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db72164-fc72-4537-a3e4-eeed32cde775 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:14.381204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:14.382069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:14.382508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:14.382937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:14.478839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:14.482025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:14.503437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1718] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:14.503683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:14.505327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:14.505632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:14.507160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:14.516798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 869/1719] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:14.517864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:14.530871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab18cf-b829-44be-91e3-e7b88b13d9f1 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:14.539054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83712868-9cbb-4898-bd72-29aac59e0c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:14.539798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83712868-9cbb-4898-bd72-29aac59e0c35 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:14.540158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83712868-9cbb-4898-bd72-29aac59e0c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:14.540540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83712868-9cbb-4898-bd72-29aac59e0c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:14.645545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5451d40-cbb4-4363-aeb5-36cf5a353fd4 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:14.657242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:14.659564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:14.659564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:14.660412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83712868-9cbb-4898-bd72-29aac59e0c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:14.662065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83712868-9cbb-4898-bd72-29aac59e0c35 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:14.663061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:14.666721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:14.686681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 870/1720] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:14.690866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:14.692869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f4624-f42a-444f-85aa-598f79d5118c service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:14.706400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1721] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:15.214160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:15.214461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:15.214754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:15.243987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] MFA Rules not processed for user `62766e7afd01401293df7e08707d6d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:15.245303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:15.293796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b28500-5aff-434c-ae32-3761322e8f70 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:15.294283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b28500-5aff-434c-ae32-3761322e8f70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:15.294374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b28500-5aff-434c-ae32-3761322e8f70 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:15.296103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1722] 10.222.0.13 () {54 vars in 912 bytes} [Tue Jul 19 12:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:15.305742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:15.306040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:15.306351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:15.367543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:15.368829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:15.471321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 871/1723] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:15.541641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1724] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:15.557144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:15.570375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf915b-401b-44e9-aad0-ca0c2fdfdb7e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:15.579839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd03b36-6729-4c36-8689-dabb2d6b16d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:15.580931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03b36-6729-4c36-8689-dabb2d6b16d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:15.581400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03b36-6729-4c36-8689-dabb2d6b16d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:15.581880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd03b36-6729-4c36-8689-dabb2d6b16d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:15.607990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd03b36-6729-4c36-8689-dabb2d6b16d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:15.610102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd03b36-6729-4c36-8689-dabb2d6b16d0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:15.620384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 872/1725] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:16.633291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:16.636647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:16.669579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:16.888194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820f078-7e07-4641-ab3b-e1e3f18c0626 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:16.903219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:16.906635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:16.907653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:16.908703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] PATH_INFO: `/v3/users/b1b1566726794259a515728a31a24a85` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:16.912944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b1566726794259a515728a31a24a85)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:16.915679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:31:16.915679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 19 12:31:16.915679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: Jul 19 12:31:16.918513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:17.080427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b1566726794259a515728a31a24a85'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:17.081705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b1566726794259a515728a31a24a85'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:17.098834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b1566726794259a515728a31a24a85'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:17.130128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b1566726794259a515728a31a24a85'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:17.175379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b1566726794259a515728a31a24a85'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:17.193261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1726] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:16 2022] DELETE /identity/v3/users/b1b1566726794259a515728a31a24a85 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:17.199733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:17.200470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:17.200470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:17.249859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:17.251942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:17.281618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:17.400760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 873/1727] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:17.411541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:17.413268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:17.561211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c53ef3-6d13-44d5-9a59-7b4c20d9dba1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:17.576668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:17.577351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:17.577828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:17.578121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:17.611810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:17.615841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:17.646301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1728] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:18.706971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:18.709429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:18.712461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:18.715804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:18.750812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99ffad-e9a6-48de-b7b9-310b339ba179 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:18.755069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4070b4a-46b8-4e74-b2aa-95027a2c2b3e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:18.765926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:18.766321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:18.767022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:18.768053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:18.768451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] PATH_INFO: `/v3/users/5f916c323dd942c1ac42c8dd6d42d552` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:18.770160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f916c323dd942c1ac42c8dd6d42d552)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:18.770669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:18.771267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:18.773542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:18.776034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] PATH_INFO: `/v3/projects/402f66e81fd14db9ac98506798ba4282` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:18.779056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=402f66e81fd14db9ac98506798ba4282)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:18.782455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: WARNING py.warnings [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 19 12:31:18.782455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: warnings.warn(msg) Jul 19 12:31:18.782455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: Jul 19 12:31:18.786220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:18.811378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:18.855486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f916c323dd942c1ac42c8dd6d42d552'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:18.856645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f916c323dd942c1ac42c8dd6d42d552'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:18.870230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f916c323dd942c1ac42c8dd6d42d552'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:18.884391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f916c323dd942c1ac42c8dd6d42d552'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:18.933794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77393226-bbb7-4e97-bd68-f2b5474db96f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f916c323dd942c1ac42c8dd6d42d552'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:18.948780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1729] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:18 2022] DELETE /identity/v3/users/5f916c323dd942c1ac42c8dd6d42d552 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:18.953289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:18.953761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:18.954324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:18.971995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '402f66e81fd14db9ac98506798ba4282', 'request_id': 'req-b9019b1f-1da0-4259-8cfc-1ad0511dced9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:18.991931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '402f66e81fd14db9ac98506798ba4282', 'request_id': 'req-b9019b1f-1da0-4259-8cfc-1ad0511dced9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.010245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:19.011956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.021377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '402f66e81fd14db9ac98506798ba4282', 'request_id': 'req-b9019b1f-1da0-4259-8cfc-1ad0511dced9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.052303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] The token cache is being invalidate because project 402f66e81fd14db9ac98506798ba4282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:19.052628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.053386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-b9019b1f-1da0-4259-8cfc-1ad0511dced9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.060327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 874/1730] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:18 2022] DELETE /identity/v3/projects/402f66e81fd14db9ac98506798ba4282 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:19.063195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01261efb-5377-4e72-a5a1-fdd70048abe4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.063480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01261efb-5377-4e72-a5a1-fdd70048abe4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.063770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01261efb-5377-4e72-a5a1-fdd70048abe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.095245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-01261efb-5377-4e72-a5a1-fdd70048abe4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:19.096675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-01261efb-5377-4e72-a5a1-fdd70048abe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.107120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 875/1731] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:19.112672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1732] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:19.117128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.117528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.117889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.120152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.125024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.148509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:19.149653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.159660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 876/1733] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:19.164154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.177892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204d60-275a-4d2f-8f9d-13fdafe3a6bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.185671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.186864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.187559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.187993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.190105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:19.191046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.193204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.208104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205bfbe-0c63-4cff-b43e-ef1bdee7b2c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.211491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 877/1734] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:19.213012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.214887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.215551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.215876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.216170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.223847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00f36d-54d7-4a0d-b41a-85a169a09f8e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.227433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.228790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.228997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.229459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.229748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.230074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.230828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.232888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.242728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1735] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:19.264002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 878/1736] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:19 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:19.268825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.280091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bdc56-2684-4f51-af8c-3f55a07327f2 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.338662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:19.358127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.360276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.360630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.360948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.361956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.364796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.397954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1737] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:19 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:19.402673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.419483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a525a7-a4c6-44ba-9f16-48ac55f200d6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.451601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.452623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.453056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.453479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.455035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.459146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.478426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 879/1738] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:19.482724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.494079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c7490-5487-4d28-b0f9-336ad08115ed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.516591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.517859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.518311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.518596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.519336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.521015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.538124 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1739] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:19.542545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.556609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09243de8-12b8-4545-9ddd-44a2efc39858 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.563725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db670a5-a23a-4e8b-a55e-cf82b403ced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.565377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db670a5-a23a-4e8b-a55e-cf82b403ced6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.565665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db670a5-a23a-4e8b-a55e-cf82b403ced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.565951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db670a5-a23a-4e8b-a55e-cf82b403ced6 admin admin] PATH_INFO: `/v3/projects/f786a4af41504096919b50d3ccd94e82/users/d26bedbd41de4f0fac9fdfe2ecc0a2ce/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.595907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.597812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.599709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db670a5-a23a-4e8b-a55e-cf82b403ced6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f786a4af41504096919b50d3ccd94e82, user_id=d26bedbd41de4f0fac9fdfe2ecc0a2ce, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.609141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db670a5-a23a-4e8b-a55e-cf82b403ced6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.615367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56805fc3-3033-4cee-981e-c128873d0365 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.628792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.629687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.630529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.630826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] PATH_INFO: `/v3/projects/f43165885aec454c8f72eb44b5d8ba62` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.636367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43165885aec454c8f72eb44b5d8ba62)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.641484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.647791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 880/1740] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:19 2022] PUT /identity/v3/projects/f786a4af41504096919b50d3ccd94e82/users/d26bedbd41de4f0fac9fdfe2ecc0a2ce/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:19.659142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.659899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.661043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.703813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] MFA Rules not processed for user `d26bedbd41de4f0fac9fdfe2ecc0a2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:19.706069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.713582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:19.727079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43165885aec454c8f72eb44b5d8ba62', 'request_id': 'req-a7dec041-264b-4a68-b298-afab5f0e5026'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.733141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43165885aec454c8f72eb44b5d8ba62', 'request_id': 'req-a7dec041-264b-4a68-b298-afab5f0e5026'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.742336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43165885aec454c8f72eb44b5d8ba62', 'request_id': 'req-a7dec041-264b-4a68-b298-afab5f0e5026'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.755847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] The token cache is being invalidate because project f43165885aec454c8f72eb44b5d8ba62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:19.756121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.756902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.758645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1741] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:19 2022] DELETE /identity/v3/projects/f43165885aec454c8f72eb44b5d8ba62 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:19.762385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.763835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.784499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 881/1742] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:19.795128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:19.796564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:19.828668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dec041-264b-4a68-b298-afab5f0e5026 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.833938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.834807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.835095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.835373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] PATH_INFO: `/v3/users/62d7825cd59e4987ae313da1ba27bc1d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.836670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d7825cd59e4987ae313da1ba27bc1d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:19.838585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:19.871871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4c03f-3cd3-4977-beac-befd155371ec None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:19.884262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d7825cd59e4987ae313da1ba27bc1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.884933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d7825cd59e4987ae313da1ba27bc1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.888550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:19.894038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d7825cd59e4987ae313da1ba27bc1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.901661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d7825cd59e4987ae313da1ba27bc1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.914958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d7825cd59e4987ae313da1ba27bc1d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:19.921198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1743] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:19 2022] DELETE /identity/v3/users/62d7825cd59e4987ae313da1ba27bc1d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:19.937022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:19.937809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:19.938097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:19.938278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:19.940606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:20.002492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:20.036306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.036306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa0c5a-af5f-409b-9a9e-a4cbf9ebf721 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.043420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 882/1744] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:20.275590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.277593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:20.367763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ef9a-436d-4349-a6c3-6c555f162f1c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.372997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.374238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.374723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.374955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] PATH_INFO: `/v3/projects/718bcf6ce0b54c2f915e76bdf6a67a72` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.376491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=718bcf6ce0b54c2f915e76bdf6a67a72)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.379139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.438404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718bcf6ce0b54c2f915e76bdf6a67a72', 'request_id': 'req-c252f025-d585-4981-af6f-2cbc3d16ce6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:20.452495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718bcf6ce0b54c2f915e76bdf6a67a72', 'request_id': 'req-c252f025-d585-4981-af6f-2cbc3d16ce6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:20.485687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718bcf6ce0b54c2f915e76bdf6a67a72', 'request_id': 'req-c252f025-d585-4981-af6f-2cbc3d16ce6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:20.501315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] The token cache is being invalidate because project 718bcf6ce0b54c2f915e76bdf6a67a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:20.501705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:20.502372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c252f025-d585-4981-af6f-2cbc3d16ce6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:20.503914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1745] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:20 2022] DELETE /identity/v3/projects/718bcf6ce0b54c2f915e76bdf6a67a72 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:20.508762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.509054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.509338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.527277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:20.528503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:20.593824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 883/1746] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:20.602277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.602574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.602859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.624647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:20.625842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:20.634420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1747] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:20.638564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.648292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba57e7e-7438-4ac6-b974-79b4f415cbf7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.652742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.653394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.653678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.653955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.654676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:20.655002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.656620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.668639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 884/1748] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:20.672753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.683267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57714923-2be9-45e4-b2fa-9f362eab400e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.687819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.688526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.688815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.689126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.689831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.691724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.716930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1749] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:20 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:20.721719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.732005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e63763-e354-4ef7-8ddd-39c3e9164762 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.788272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.788940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.789226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.789513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.790201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.792127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.811574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 885/1750] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:20 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:20.815799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.826050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b7154-0018-49c1-8eef-52f6370830f5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.830610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.831263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.831552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.831834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.832615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.834209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.844544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1751] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:20.848801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.858989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81512d1-1a2d-4630-8e85-60c1e5b10102 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.864123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e12a39-e9aa-4e1f-a0b1-b4b540cd20cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.864841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12a39-e9aa-4e1f-a0b1-b4b540cd20cb admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.865133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12a39-e9aa-4e1f-a0b1-b4b540cd20cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.865416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12a39-e9aa-4e1f-a0b1-b4b540cd20cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.866138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e12a39-e9aa-4e1f-a0b1-b4b540cd20cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.867736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e12a39-e9aa-4e1f-a0b1-b4b540cd20cb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.877786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 886/1752] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:20.881672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:20.891280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04212763-013c-4da8-a25b-446192e390cf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:20.895632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:20.896521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.896801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.897077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] PATH_INFO: `/v3/projects/04f261599d724d09867f250ee508a0dc/users/3be6148e0cdd418381c8fdf9fd37ca8d/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.911557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f261599d724d09867f250ee508a0dc, user_id=3be6148e0cdd418381c8fdf9fd37ca8d, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:20.914197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:20.916362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:20.928519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1753] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:20 2022] PUT /identity/v3/projects/04f261599d724d09867f250ee508a0dc/users/3be6148e0cdd418381c8fdf9fd37ca8d/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:20.940121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d547ae0-e133-49ac-b0ed-9832e29bc91e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:20.940442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d547ae0-e133-49ac-b0ed-9832e29bc91e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:20.940745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d547ae0-e133-49ac-b0ed-9832e29bc91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:20.966650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d547ae0-e133-49ac-b0ed-9832e29bc91e None None] MFA Rules not processed for user `3be6148e0cdd418381c8fdf9fd37ca8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:20.967870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d547ae0-e133-49ac-b0ed-9832e29bc91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:21.002127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:21.034673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 887/1754] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:21.041148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.067441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4426e5-aa86-43de-9f16-14a812c6664a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.067891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4426e5-aa86-43de-9f16-14a812c6664a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.069719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4426e5-aa86-43de-9f16-14a812c6664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.096343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ea4426e5-aa86-43de-9f16-14a812c6664a None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:21.097772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ea4426e5-aa86-43de-9f16-14a812c6664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:21.180126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81c6f4-8216-4dc1-ac47-0f473b8ede82 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.191572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.192378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.192835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.195290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.196324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:21.199065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:21.201751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 888/1755] 10.222.0.42 () {58 vars in 1015 bytes} [Tue Jul 19 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:21.208338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.216109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.216465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.262512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1756] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:21 2022] POST /identity/v3/projects => generated 379 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:21.263628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.293802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:21.337771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:21.340100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:21.398911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db8a8-760a-4b82-aec9-29760f75c79f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.410489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.411085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.412058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.412058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.422733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:21.424446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:21.434433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1757] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:21.449328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.464058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 889/1758] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:21.602547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7b27f-eee6-469c-8e70-c9d337675938 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.619882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.620627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.620914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.621189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.621896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:21.623853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:21.628514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.630028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:21.680097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1759] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:21 2022] POST /identity/v3/users => generated 296 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:21.681361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.705114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765b11-8ad7-4818-bc6d-200c73b86b2b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.721252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.722202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.722639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.723236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.723978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:21.725929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:21.732697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:21.733593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:21.745480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1760] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:21.747097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.762017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaee190-eccc-4355-ae2a-bca1f934d2ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.770062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.771248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.771537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.771823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] PATH_INFO: `/v3/projects/0144271cc8c2430d85c36f55ba6540e9/users/f97f58fcef5244feafdc67254ceb680e/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.793636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0144271cc8c2430d85c36f55ba6540e9, user_id=f97f58fcef5244feafdc67254ceb680e, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:21.798855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:21.814731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637c525-f485-4619-bc51-5e0609b3222d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.821092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1761] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:21 2022] PUT /identity/v3/projects/0144271cc8c2430d85c36f55ba6540e9/users/f97f58fcef5244feafdc67254ceb680e/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:21.825302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.936149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff64c4e-edff-4c54-abbc-42efaaa0fa37 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.948051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.948781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.949072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.949359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.950576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:21.952428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:21.967672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1762] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:21.969072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:21.982327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a613d38-b08b-440a-bab7-2ef8f9ea51bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:21.994339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bc407-b5b2-4e2f-b9ef-1f19f5d6b806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.994546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:21.995041 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.995329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.995635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:21.996545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bc407-b5b2-4e2f-b9ef-1f19f5d6b806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:21.997068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bc407-b5b2-4e2f-b9ef-1f19f5d6b806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:21.997496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bc407-b5b2-4e2f-b9ef-1f19f5d6b806 admin admin] PATH_INFO: `/v3/projects/0144271cc8c2430d85c36f55ba6540e9/users/f97f58fcef5244feafdc67254ceb680e/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:22.000962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bc407-b5b2-4e2f-b9ef-1f19f5d6b806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0144271cc8c2430d85c36f55ba6540e9, user_id=f97f58fcef5244feafdc67254ceb680e, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:22.004784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bc407-b5b2-4e2f-b9ef-1f19f5d6b806 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:22.019219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1763] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:21 2022] PUT /identity/v3/projects/0144271cc8c2430d85c36f55ba6540e9/users/f97f58fcef5244feafdc67254ceb680e/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:22.029936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d8422-c093-483f-902d-ce651fb3f96a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:22.030372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d8422-c093-483f-902d-ce651fb3f96a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:22.030665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d8422-c093-483f-902d-ce651fb3f96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:22.064541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-317d8422-c093-483f-902d-ce651fb3f96a None None] MFA Rules not processed for user `f97f58fcef5244feafdc67254ceb680e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:22.065917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-317d8422-c093-483f-902d-ce651fb3f96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:22.085802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:22.087557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:22.119364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 890/1764] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:31:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 12:31:22.231248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1765] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:22.254313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:22.260574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:22.415049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4801bc-52f7-4014-ac7c-3e171b1605db service swift] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:22.524517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:22.591455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2acfcb3-426d-49d0-a072-a5f97dea2704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:22.593518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acfcb3-426d-49d0-a072-a5f97dea2704 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:22.593873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acfcb3-426d-49d0-a072-a5f97dea2704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:22.594163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acfcb3-426d-49d0-a072-a5f97dea2704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:22.644854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acfcb3-426d-49d0-a072-a5f97dea2704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:22.647400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2acfcb3-426d-49d0-a072-a5f97dea2704 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:22.693267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 891/1766] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:23.140416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516e78a-9f0a-4ca1-9ebe-9cc9b918c47d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:23.140416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516e78a-9f0a-4ca1-9ebe-9cc9b918c47d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:23.141062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516e78a-9f0a-4ca1-9ebe-9cc9b918c47d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:23.142196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1767] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:23.224450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:23.225292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:23.226735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:23.306440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:23.309123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:23.544405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 892/1768] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:23.728075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516e78a-9f0a-4ca1-9ebe-9cc9b918c47d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:23.728075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e516e78a-9f0a-4ca1-9ebe-9cc9b918c47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:23.743785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:23.747708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:23.822917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866fbf2-fae2-4395-8f10-cd379489c796 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:23.851790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:23.851790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:23.851790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:23.851790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:23.883489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:23.885380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:23.934893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 893/1769] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:23.946929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:24.040976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516e78a-9f0a-4ca1-9ebe-9cc9b918c47d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:24.047709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:24.048997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:24.049450 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:24.049958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] PATH_INFO: `/v3/users/d26bedbd41de4f0fac9fdfe2ecc0a2ce` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:24.055971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26bedbd41de4f0fac9fdfe2ecc0a2ce)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:24.064996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:24.226733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26bedbd41de4f0fac9fdfe2ecc0a2ce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.232618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26bedbd41de4f0fac9fdfe2ecc0a2ce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.247626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26bedbd41de4f0fac9fdfe2ecc0a2ce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.274698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26bedbd41de4f0fac9fdfe2ecc0a2ce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.301690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:24.303471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:24.333016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2a93423c-9c70-4736-9cb4-a17b06d0e5c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26bedbd41de4f0fac9fdfe2ecc0a2ce'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.347079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1770] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:23 2022] DELETE /identity/v3/users/d26bedbd41de4f0fac9fdfe2ecc0a2ce => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:24.353160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:24.353282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:24.353477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:24.356255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0480f-1954-4b6f-84b6-57f5865d9a91 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:24.405369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:24.409380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:24.540768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:24.541663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:24.541943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:24.545616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] PATH_INFO: `/v3/users/df2c1f75eb414f869c54a6c2b1ab416d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:24.545918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df2c1f75eb414f869c54a6c2b1ab416d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:24.554341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:24.601014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1771] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:24.726886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df2c1f75eb414f869c54a6c2b1ab416d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.742358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:24.744201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:24.749752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df2c1f75eb414f869c54a6c2b1ab416d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.751424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df2c1f75eb414f869c54a6c2b1ab416d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.766046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:24.797164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df2c1f75eb414f869c54a6c2b1ab416d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.809068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f79d6785-476f-4417-9b23-76b76b3e97c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df2c1f75eb414f869c54a6c2b1ab416d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:24.816966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 894/1772] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:24 2022] DELETE /identity/v3/users/df2c1f75eb414f869c54a6c2b1ab416d => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:24.822347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:24.822872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:24.824317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:24.885845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:24.887538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:24.907001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7c327-c5e3-4127-9826-9ac86af1750d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:24.944050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:25.062666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:25.064661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:25.065136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:25.066403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:25.071633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:25.098150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 895/1773] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:25.099098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:25.100461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:25.114100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b12a9-4fc4-47b0-a79e-c0f19176cfe4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:25.124264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:25.125212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:25.125635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:25.125921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] PATH_INFO: `/v3/users/3be6148e0cdd418381c8fdf9fd37ca8d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:25.127861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be6148e0cdd418381c8fdf9fd37ca8d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:25.129894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:25.228094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be6148e0cdd418381c8fdf9fd37ca8d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:25.243491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be6148e0cdd418381c8fdf9fd37ca8d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:25.244614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be6148e0cdd418381c8fdf9fd37ca8d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:25.267088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:25.270429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be516c1-870b-43d0-80da-e1802fc235ca service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:25.272502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be6148e0cdd418381c8fdf9fd37ca8d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:25.286350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be6148e0cdd418381c8fdf9fd37ca8d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:25.287594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1774] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:25.301887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 896/1775] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:25 2022] DELETE /identity/v3/users/3be6148e0cdd418381c8fdf9fd37ca8d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:25.303048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:25.303339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:25.303789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:25.343253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:25.347181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:25.396030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:25.398560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:25.592119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adbd06-055c-4cf6-a4ff-e939d71cf996 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:25.598025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1776] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:25.599174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:25.600387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:25.600860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:25.601285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:25.605406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:25.633912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:25.654402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:25.656479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:25.700900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 897/1777] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:25.797719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:25.800006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:25.847769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d2cba-942e-49a4-b1ff-666d48cd0b42 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:25.862796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6abe00-7980-4de7-87fb-02b7b829f40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:25.867532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6abe00-7980-4de7-87fb-02b7b829f40e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:25.869882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6abe00-7980-4de7-87fb-02b7b829f40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:25.871281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6abe00-7980-4de7-87fb-02b7b829f40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:25.903189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6abe00-7980-4de7-87fb-02b7b829f40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:25.918275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6abe00-7980-4de7-87fb-02b7b829f40e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:25.957522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1778] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:27.090202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:27.091900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:27.112212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65656-0359-48c2-8b1e-d07fab5f212b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:27.112212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65656-0359-48c2-8b1e-d07fab5f212b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:27.112901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b65656-0359-48c2-8b1e-d07fab5f212b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:27.114267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1779] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:27.120888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070a100-4362-4738-99e2-97087a0a73da service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:27.137448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:27.138591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:27.138925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:27.139312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] PATH_INFO: `/v3/users/8a31429b757c4d8baf4d8fbf62218872` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:27.140809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a31429b757c4d8baf4d8fbf62218872)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:27.143738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:27.146320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:27.146874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:27.146874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:27.211292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:27.211292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:27.280291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a31429b757c4d8baf4d8fbf62218872'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:27.306111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a31429b757c4d8baf4d8fbf62218872'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:27.312211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a31429b757c4d8baf4d8fbf62218872'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:27.355530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a31429b757c4d8baf4d8fbf62218872'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:27.375809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a888c62c-588f-48c0-a010-7be3f05d8db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a31429b757c4d8baf4d8fbf62218872'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:27.390061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 898/1780] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:27 2022] DELETE /identity/v3/users/8a31429b757c4d8baf4d8fbf62218872 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:27.394888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:27.395572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:27.396010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:27.433411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:27.441383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:27.442863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:27.535196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1781] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:27.544344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:27.558249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:27.619051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983393c4-745e-4970-bc61-564d05f0b1ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:27.795509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 899/1782] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:27.889025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:27.890183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:27.890183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:27.890544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] PATH_INFO: `/v3/projects/f786a4af41504096919b50d3ccd94e82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:27.907241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f786a4af41504096919b50d3ccd94e82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:27.921653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:27.944964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:27.946530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.047442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f786a4af41504096919b50d3ccd94e82', 'request_id': 'req-042e7d40-a288-4feb-a31a-3bd6edeaf283'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.057912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f786a4af41504096919b50d3ccd94e82', 'request_id': 'req-042e7d40-a288-4feb-a31a-3bd6edeaf283'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.069793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f786a4af41504096919b50d3ccd94e82', 'request_id': 'req-042e7d40-a288-4feb-a31a-3bd6edeaf283'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.098864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] The token cache is being invalidate because project f786a4af41504096919b50d3ccd94e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:28.099325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.102866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.110447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1783] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:27 2022] DELETE /identity/v3/projects/f786a4af41504096919b50d3ccd94e82 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:28.111883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:28.116074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.180385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be058e86-0259-42f6-9267-1cbb02b6586f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:28.268918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:28.386172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:28.389427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:28.390014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:28.391867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:28.399241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.414549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e7d40-a288-4feb-a31a-3bd6edeaf283 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:28.439773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:28.440705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:28.441117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:28.441400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:28.441579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] PATH_INFO: `/v3/projects/04f261599d724d09867f250ee508a0dc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:28.442890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd0f0d-1393-4f12-b26b-1a4e01d566bd service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:28.449649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f261599d724d09867f250ee508a0dc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:28.451640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:28.459118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 900/1784] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:28.463289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:28.464402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:28.464908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:28.484830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:28.507841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:28.512915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.558030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f261599d724d09867f250ee508a0dc', 'request_id': 'req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.577236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f261599d724d09867f250ee508a0dc', 'request_id': 'req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.593491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f261599d724d09867f250ee508a0dc', 'request_id': 'req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.644413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] The token cache is being invalidate because project 04f261599d724d09867f250ee508a0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:28.644874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.656661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.662103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1785] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:28 2022] DELETE /identity/v3/projects/04f261599d724d09867f250ee508a0dc => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:28.663191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:28.670281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 901/1786] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:28.671609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:28.672075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.684027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.707899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59ee5b-41cb-42d9-81cf-4676fa1a9592 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:28.713928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:28.714985 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:28.715398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:28.715763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] PATH_INFO: `/v3/projects/c1a6059c78164a06ab14f53c41962199` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:28.717283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a6059c78164a06ab14f53c41962199)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:28.719554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:28.721165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee368e7c-20bd-44c1-bc15-4e8895b04984 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:28.735391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:28.741524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:28.743011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:28.743633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] PATH_INFO: `/v3/users/f97f58fcef5244feafdc67254ceb680e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:28.746744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f97f58fcef5244feafdc67254ceb680e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:28.756438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:28.783218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a6059c78164a06ab14f53c41962199', 'request_id': 'req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.790209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a6059c78164a06ab14f53c41962199', 'request_id': 'req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.801832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a6059c78164a06ab14f53c41962199', 'request_id': 'req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.822484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] The token cache is being invalidate because project c1a6059c78164a06ab14f53c41962199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:28.822830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.824137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71ee6fe5-fb6b-4b47-b882-03a78cd379e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.826855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1787] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:28 2022] DELETE /identity/v3/projects/c1a6059c78164a06ab14f53c41962199 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:28.828149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2baf05-9c85-46fa-9bee-862505da1084 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:28.828533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2baf05-9c85-46fa-9bee-862505da1084 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:28.828885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2baf05-9c85-46fa-9bee-862505da1084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:28.866703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff2baf05-9c85-46fa-9bee-862505da1084 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:28.869023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff2baf05-9c85-46fa-9bee-862505da1084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:28.886523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f97f58fcef5244feafdc67254ceb680e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.923074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f97f58fcef5244feafdc67254ceb680e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.925194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f97f58fcef5244feafdc67254ceb680e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.956411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f97f58fcef5244feafdc67254ceb680e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.967544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f97f58fcef5244feafdc67254ceb680e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:28.996783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 902/1788] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:28 2022] DELETE /identity/v3/users/f97f58fcef5244feafdc67254ceb680e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:29.000719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.005212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:29.023351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1789] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:29.027816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a464-17b9-4d75-8b2d-33d873772018 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.028121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a464-17b9-4d75-8b2d-33d873772018 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.029269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a464-17b9-4d75-8b2d-33d873772018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.093125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:29.093810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5fd9a464-17b9-4d75-8b2d-33d873772018 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:29.096541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5fd9a464-17b9-4d75-8b2d-33d873772018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:29.174286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bf081-8943-4a5e-9b50-289b9437e9d2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.185635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.188015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.189601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.189987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.195342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:29.195844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:29.215314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.218027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.245007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 903/1790] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:29.249302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.305582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1791] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:29.314746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.315096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.315341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.373638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dda53-dbd0-48e6-b538-70930954c85d service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.380518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.381998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.382300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.382582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.383572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:29.384933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.388142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.427082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 904/1792] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:29.428469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:29.429877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:29.432852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.449325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889e84c-ed85-43e7-8e54-95dcb128ac21 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.457358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1793] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:29.458363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.475090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.475849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.476326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.476646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.477497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.479603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.484487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a161e0-2b1c-4075-94d5-3fca0d8b6ace None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.494991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.495729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.496008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.496316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.497049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:29.497367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.499502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.530710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1794] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:29.535923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.554637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df4c70-7eb9-416f-98ba-aca3e6bc0687 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.564973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.565606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.565946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.567775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.567775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.569251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.571157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 905/1795] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:29 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:29.577378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.648055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1796] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:29 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:29.652137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.797409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d37fb-a7dc-4cbc-86e3-a15efe7c483e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.806205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.806949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.807249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.807583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.808343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.810303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.864270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1797] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:29 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:29.865184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.870313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:29.903254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60e5b8-0963-4a87-b217-cd69b3ee51f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.906063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7a51-f316-4335-9f06-c12757c4f5c3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:29.920271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.920271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.920271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.920271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] PATH_INFO: `/v3/projects/a9106d1dd4b3413ab60d170187627d54` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.921457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9106d1dd4b3413ab60d170187627d54)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.923723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:29.924846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:29.925350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:29.926026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:29.926922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:29.929140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.929852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:29.973601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 906/1798] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:29 2022] POST /identity/v3/users => generated 282 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:29.977399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:29.994627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcad7ab-cf0c-4dae-ae88-9d13f61e0dca admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.004137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.005965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.006475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.007668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.008660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.010721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.028919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9106d1dd4b3413ab60d170187627d54', 'request_id': 'req-2b9ab427-6884-4eeb-861d-cf5da255986d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.030417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 907/1799] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:30.032261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:30.041947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9106d1dd4b3413ab60d170187627d54', 'request_id': 'req-2b9ab427-6884-4eeb-861d-cf5da255986d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.051155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f838f-e01d-47c6-a38a-63b48c6dca53 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.058088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9106d1dd4b3413ab60d170187627d54', 'request_id': 'req-2b9ab427-6884-4eeb-861d-cf5da255986d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.075331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:30.099443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] The token cache is being invalidate because project a9106d1dd4b3413ab60d170187627d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:30.099806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.104812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.106499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1800] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:29 2022] DELETE /identity/v3/projects/a9106d1dd4b3413ab60d170187627d54 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:30.110375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:30.112486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:30.147493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:30.225650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.226681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.237362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.237974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.238878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.242045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.256935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ab427-6884-4eeb-861d-cf5da255986d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.262812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 908/1801] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:30.262812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:30.264284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:30.266252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.267647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.268089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.268533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.269428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.271323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.281736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c81b34-164b-4f6c-95d9-0372e02b9210 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.284286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1802] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:30.285336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:30.289136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:30.293502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.294792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.295860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.296793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] PATH_INFO: `/v3/users/0781a9fc92a64a2088b00c1e1a525903` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.299818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0781a9fc92a64a2088b00c1e1a525903)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.302973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.305550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997d4b0-a5e7-4b4d-9325-1b131b01a1e5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.315502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.316679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.317107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.317538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] PATH_INFO: `/v3/projects/0144271cc8c2430d85c36f55ba6540e9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.318969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0144271cc8c2430d85c36f55ba6540e9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.321153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.380077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0781a9fc92a64a2088b00c1e1a525903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.385002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0144271cc8c2430d85c36f55ba6540e9', 'request_id': 'req-ae3eba72-f267-4180-89e1-d0bad95feff6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.394875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0144271cc8c2430d85c36f55ba6540e9', 'request_id': 'req-ae3eba72-f267-4180-89e1-d0bad95feff6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.395865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0781a9fc92a64a2088b00c1e1a525903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.396758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0781a9fc92a64a2088b00c1e1a525903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.406721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0144271cc8c2430d85c36f55ba6540e9', 'request_id': 'req-ae3eba72-f267-4180-89e1-d0bad95feff6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.426848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0781a9fc92a64a2088b00c1e1a525903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.438425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] The token cache is being invalidate because project 0144271cc8c2430d85c36f55ba6540e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:30.440507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.441904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.443897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1803] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:30 2022] DELETE /identity/v3/projects/0144271cc8c2430d85c36f55ba6540e9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:30.445185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:30.446523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:30.452347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0781a9fc92a64a2088b00c1e1a525903'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:30.465843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 909/1804] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:30 2022] DELETE /identity/v3/users/0781a9fc92a64a2088b00c1e1a525903 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:30.468393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:30.470531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:30.559093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3eba72-f267-4180-89e1-d0bad95feff6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.654267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3815e-4380-4088-ac00-db3fc571c367 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:30.668203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:30.668342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd419ce3-484e-40e8-8501-13cd4202a42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.669149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd419ce3-484e-40e8-8501-13cd4202a42d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.669437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd419ce3-484e-40e8-8501-13cd4202a42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.670545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd419ce3-484e-40e8-8501-13cd4202a42d admin admin] PATH_INFO: `/v3/projects/32ebb35f5b164f3ebf13b98fd8a46273/users/1b3b49092d254f78b3bcc8d708334d94/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.733579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd419ce3-484e-40e8-8501-13cd4202a42d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ebb35f5b164f3ebf13b98fd8a46273, user_id=1b3b49092d254f78b3bcc8d708334d94, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.745820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd419ce3-484e-40e8-8501-13cd4202a42d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.788326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a5df9-6cc5-4e9a-8478-b8d3dcfb6659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:30.800428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5df9-6cc5-4e9a-8478-b8d3dcfb6659 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.800704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5df9-6cc5-4e9a-8478-b8d3dcfb6659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.800986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a5df9-6cc5-4e9a-8478-b8d3dcfb6659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.811297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a5df9-6cc5-4e9a-8478-b8d3dcfb6659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:30.816338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 910/1805] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:30 2022] PUT /identity/v3/projects/32ebb35f5b164f3ebf13b98fd8a46273/users/1b3b49092d254f78b3bcc8d708334d94/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:30.822424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a5df9-6cc5-4e9a-8478-b8d3dcfb6659 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:30.835882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.836210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.836579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.858802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1806] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:30.860393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:30.860930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359ffaa-842e-46e5-8a9a-fd7d4bcfcc74 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:30.861513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359ffaa-842e-46e5-8a9a-fd7d4bcfcc74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:30.861806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359ffaa-842e-46e5-8a9a-fd7d4bcfcc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:30.886306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:30.887524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:30.926928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d359ffaa-842e-46e5-8a9a-fd7d4bcfcc74 None None] MFA Rules not processed for user `1b3b49092d254f78b3bcc8d708334d94`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:30.928663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d359ffaa-842e-46e5-8a9a-fd7d4bcfcc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.031371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 911/1807] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:31.031371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:31.039516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cbee7-a3f9-4e45-bf01-8c9773df96bf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:31.049387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:31.050401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.050750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.051108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] PATH_INFO: `/v3/projects/29bf653d38d64a8c80a02160bf1f3fd6/users/b486b8536d1045cfab9d3ebb4c51df54/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.072855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29bf653d38d64a8c80a02160bf1f3fd6, user_id=b486b8536d1045cfab9d3ebb4c51df54, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:31.078712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:31.097396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1808] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:31.099212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f39a-5753-4282-9d0b-7a52a8b1feb5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.099706 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f39a-5753-4282-9d0b-7a52a8b1feb5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.100169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f39a-5753-4282-9d0b-7a52a8b1feb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.120006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 912/1809] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:31 2022] PUT /identity/v3/projects/29bf653d38d64a8c80a02160bf1f3fd6/users/b486b8536d1045cfab9d3ebb4c51df54/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:31.122401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:31.129628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b8b2f39a-5753-4282-9d0b-7a52a8b1feb5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:31.130769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.131417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8b2f39a-5753-4282-9d0b-7a52a8b1feb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.291113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2316e-723d-4f33-b8de-1dcfc7b03cb5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:31.303184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:31.304794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.306024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.306344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.307547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1810] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:31.309547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.309838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.310120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.342151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] MFA Rules not processed for user `b486b8536d1045cfab9d3ebb4c51df54`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:31.343357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.423416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:31.425621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:31.439505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:31.439799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 913/1811] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:31.440826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:31.442303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.459563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b877c0f-b3f3-4bc2-b798-7ce98f923f28 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:31.469164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:31.471882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.472188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.477920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] PATH_INFO: `/v3/projects/2e45b2129b684e6899457aa3b3e4dd90` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.477920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e45b2129b684e6899457aa3b3e4dd90)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:31.484700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:31.503771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:31.529535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1812] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:31.529535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:31.557699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa49b88-1f3f-443b-bb12-a2cc32a58076 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:31.599217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e45b2129b684e6899457aa3b3e4dd90', 'request_id': 'req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:31.633255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e45b2129b684e6899457aa3b3e4dd90', 'request_id': 'req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:31.644215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e45b2129b684e6899457aa3b3e4dd90', 'request_id': 'req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:31.665322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] The token cache is being invalidate because project 2e45b2129b684e6899457aa3b3e4dd90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:31.665664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:31.666420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:31.668527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 914/1813] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:31 2022] DELETE /identity/v3/projects/2e45b2129b684e6899457aa3b3e4dd90 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:31.669330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:31.670898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.737110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:31.737869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.738231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.738587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.739398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:31.739827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:31.741742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0dbde-43eb-4ee4-a0c0-0758e7514d74 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:31.756781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1814] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:31.758248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.758638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.759037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.788399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:31.789669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.803470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1815] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:31.804626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:31.806221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:31.842031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c958a-20a8-4356-9bcc-0903e76282bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:31.872404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a41825-1449-4fe3-926d-23963d73e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:31.873090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a41825-1449-4fe3-926d-23963d73e727 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.873473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a41825-1449-4fe3-926d-23963d73e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.873659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a41825-1449-4fe3-926d-23963d73e727 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.874768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a41825-1449-4fe3-926d-23963d73e727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:31.877823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a41825-1449-4fe3-926d-23963d73e727 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:31.942800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960eaeb-c307-45ae-ba06-4e48b7bd60e8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:31.955522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:31.956920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.957277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.957507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.960833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1816] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:31 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:31.960833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:31.960833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:31.961246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:31.961320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:31.964924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:31.991859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:31.993491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.030261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 915/1817] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:31 2022] POST /identity/v3/projects => generated 365 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:32.031983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.035533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.093012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1818] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:32.094192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.108809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befa880-58f7-4a63-8dfd-66322d3c22ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.176638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1d135-f33e-4566-90ff-2d0c6610c78a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.185799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.186698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.186988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.187275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] PATH_INFO: `/v3/users/1b3b49092d254f78b3bcc8d708334d94` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.189247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3b49092d254f78b3bcc8d708334d94)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.191537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.200950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:32.222087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.222998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.223461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.223947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.224818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.226962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.243836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:32.261639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1819] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:32 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:32.264590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.283996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a986daf-f15a-421d-a515-24a829f94d6f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.301890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3b49092d254f78b3bcc8d708334d94'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:32.302993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.303657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.304151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.304558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.306973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.315538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.322813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3b49092d254f78b3bcc8d708334d94'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:32.331418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3b49092d254f78b3bcc8d708334d94'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:32.352305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1820] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:32 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:32.353091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.361775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3b49092d254f78b3bcc8d708334d94'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:32.362500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.380378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3b49092d254f78b3bcc8d708334d94'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:32.394470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 916/1821] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:32 2022] DELETE /identity/v3/users/1b3b49092d254f78b3bcc8d708334d94 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:32.395553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.398772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88feb95-e7e3-400c-b2c2-7e2b801d6cac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.400101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.539767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9fab2-d1e6-4f62-8288-b5bc7da9fb57 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.564423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.565299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.566826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.567119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.567877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.569547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.572960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.573631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.574025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.574201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.575914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:32.576861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.581113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d670b12-0247-42bf-a30a-51621b59dfce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.602668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 917/1822] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:32.603813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.606711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1823] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:32.608025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.608346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.608632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.614729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.641691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfba07-7327-4bda-acd7-3a5e0ca4f0be admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.649962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.656647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.657029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.657221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.658063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.659716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.670979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:32.675156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.682312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 918/1824] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:32.683280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.694878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc3d40-1ee2-440a-96d1-4e3c66f11ba0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.700104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1825] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:32.701704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.702761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.703563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.703849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.704132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.707626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.711714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.722854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecddbdf-1088-427a-9d56-e2f4e8c1d134 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.730651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 919/1826] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:32.735384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.742505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.743189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.743523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.743812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.744587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.746601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.757168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49a76-9b16-442b-9956-387237c3add7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.772076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.773134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.773441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.773730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] PATH_INFO: `/v3/projects/e508aa73352348a8a1360121bca35abd/users/7ec13db808b14c9ab72cc6d7381d5678/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.795854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1827] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:32 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:32.804514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.807116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:32.820542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e508aa73352348a8a1360121bca35abd, user_id=7ec13db808b14c9ab72cc6d7381d5678, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:32.823562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:32.856271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 920/1828] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:32 2022] PUT /identity/v3/projects/e508aa73352348a8a1360121bca35abd/users/7ec13db808b14c9ab72cc6d7381d5678/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:32.857480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:32.947868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:32.956020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031179c-d2ac-45f5-b55b-3f46b430d8a2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.963214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c803d-af09-482a-9810-ea9b54b136a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:32.965656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:32.966790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:32.967176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:32.967564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] PATH_INFO: `/v3/projects/508ea692359b410b956592ee02507b0d/users/9403835e4424431fa26c09abe25ec081/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:32.997057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=508ea692359b410b956592ee02507b0d, user_id=9403835e4424431fa26c09abe25ec081, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.000130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.031370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 921/1829] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:32 2022] PUT /identity/v3/projects/508ea692359b410b956592ee02507b0d/users/9403835e4424431fa26c09abe25ec081/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:33.043670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.077575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:33.090499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.091386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.091784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.092515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.109888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.112485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.126699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1830] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:33.130882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.136724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8f65-75b0-4fc8-ab1c-dcdca43be5c5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.143644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.144814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.145249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.145735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.147017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.149247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e606e-1f72-4d39-9091-4b0802fdfaf4 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.149501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.161923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729278a-514e-42e1-a335-fb24d7766c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.162618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729278a-514e-42e1-a335-fb24d7766c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.162926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729278a-514e-42e1-a335-fb24d7766c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.163213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729278a-514e-42e1-a335-fb24d7766c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.163949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729278a-514e-42e1-a335-fb24d7766c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.166029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729278a-514e-42e1-a335-fb24d7766c63 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.183832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1831] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:33.185322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111b4c-6bd8-4822-ba44-732da3acd9d4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.185669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111b4c-6bd8-4822-ba44-732da3acd9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.185947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111b4c-6bd8-4822-ba44-732da3acd9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.190612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 922/1832] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:33 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:33.191508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.211661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b270d8-3469-4c32-9732-040b141182a2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.217231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.218201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.218490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.218768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] PATH_INFO: `/v3/projects/e508aa73352348a8a1360121bca35abd/users/7ec13db808b14c9ab72cc6d7381d5678/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.223085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e508aa73352348a8a1360121bca35abd, user_id=7ec13db808b14c9ab72cc6d7381d5678, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.229821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.238638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1c111b4c-6bd8-4822-ba44-732da3acd9d4 None None] MFA Rules not processed for user `9403835e4424431fa26c09abe25ec081`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:33.239973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c111b4c-6bd8-4822-ba44-732da3acd9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:33.250704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 923/1833] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:33 2022] PUT /identity/v3/projects/e508aa73352348a8a1360121bca35abd/users/7ec13db808b14c9ab72cc6d7381d5678/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:33.254052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.348796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6ae7-4d1c-42ed-89e4-d380ce98aa45 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.357812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.358604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.358892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.359204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.360051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.361867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.364424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1834] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:33.365922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.366216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.369469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.388950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 924/1835] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:33.390103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.392224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:33.410658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] MFA Rules not processed for user `7ec13db808b14c9ab72cc6d7381d5678`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:33.412289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:33.522514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1836] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:33.525764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.545961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae42394-2957-4741-bc24-435a1615aea0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.547958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eec8bf-3280-4567-8901-9bce82cc4f56 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.549816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.551190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.551378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.551779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.557822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.558964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:33.559181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.560431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.560929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.564964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.567579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.577007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.578586 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 925/1837] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:33.579841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.581453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:33.581930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.595824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e8bd1-5f55-48e8-ac92-eba4100bca24 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.600021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f830b6b-a180-436e-85a1-e991f30339dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.600743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830b6b-a180-436e-85a1-e991f30339dd service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.601030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830b6b-a180-436e-85a1-e991f30339dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.601313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f830b6b-a180-436e-85a1-e991f30339dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.607196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1838] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:33.614166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.618865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830b6b-a180-436e-85a1-e991f30339dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.620537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f830b6b-a180-436e-85a1-e991f30339dd service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.627378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f24ae-a5e5-4b15-a39c-99cb72b52ca4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.632485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.633525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.633817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.634099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] PATH_INFO: `/v3/projects/37ba4bda52d14d20b5fc94d3cc6c644f/users/d4e22bb199fd4fd9b799ee13c4942460/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.635700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 926/1839] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:33.655945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ba4bda52d14d20b5fc94d3cc6c644f, user_id=d4e22bb199fd4fd9b799ee13c4942460, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.677435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:33.712326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1840] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:33 2022] PUT /identity/v3/projects/37ba4bda52d14d20b5fc94d3cc6c644f/users/d4e22bb199fd4fd9b799ee13c4942460/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:33.722641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.723073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.723516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.750419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:33.753402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:33.780263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] MFA Rules not processed for user `d4e22bb199fd4fd9b799ee13c4942460`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:33.781486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:33.795830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:33.905944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5fc0e8-1de0-42a0-9ee8-561c02fc927e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:33.912541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:33.914276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:33.914796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:33.915262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] PATH_INFO: `/v3/projects/32ebb35f5b164f3ebf13b98fd8a46273` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:33.921797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ebb35f5b164f3ebf13b98fd8a46273)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:33.925180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.012240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 927/1841] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:34.014759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.036212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ebb35f5b164f3ebf13b98fd8a46273', 'request_id': 'req-691f89c1-46e7-4df5-aee3-5c82e6616a2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:34.045576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ebb35f5b164f3ebf13b98fd8a46273', 'request_id': 'req-691f89c1-46e7-4df5-aee3-5c82e6616a2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:34.049406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92bf6c-bc7d-441a-9824-8a254cd9daef None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.074636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ebb35f5b164f3ebf13b98fd8a46273', 'request_id': 'req-691f89c1-46e7-4df5-aee3-5c82e6616a2e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:34.094205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] The token cache is being invalidate because project 32ebb35f5b164f3ebf13b98fd8a46273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:34.094550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:34.095765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691f89c1-46e7-4df5-aee3-5c82e6616a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:34.097912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1842] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:33 2022] DELETE /identity/v3/projects/32ebb35f5b164f3ebf13b98fd8a46273 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:34.102541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.102988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.103243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.137432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:34.138848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:34.163404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.164204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.164532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.164957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.165850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.168611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.214344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 928/1843] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:34.215555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.217218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:34.264024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1844] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:34.265131 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.269551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:34.290552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33e089-96d1-4829-83bb-84d80b759a71 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.328882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8957cc-e0cd-4552-ad74-5e87c94185a5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.334191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.335082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.335645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.336105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.339859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:34.351392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:34.370949 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:34.371933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdc697-7a52-4db8-97f9-d64b93a8f48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.373023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc697-7a52-4db8-97f9-d64b93a8f48f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.374296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc697-7a52-4db8-97f9-d64b93a8f48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.374785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdc697-7a52-4db8-97f9-d64b93a8f48f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.375816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdc697-7a52-4db8-97f9-d64b93a8f48f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.379545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdc697-7a52-4db8-97f9-d64b93a8f48f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.423988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1845] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:34.426156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.426564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.426953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.447150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:34.448646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:34.460270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1846] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:34.461512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.468416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.469911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.481209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3300d-d637-4f86-a5cd-f3e7ca56e29f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.486289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 929/1847] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:34.487421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.488242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.488948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.489066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.489848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.491558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.495711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.505237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1848] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:34.509573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.516161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3883f6-1634-444a-8aab-aafdb1e9f0e7 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.521561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.522451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.522666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.522942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.523662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:34.523969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.525015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2cdf8-2ae2-4589-b53f-b92db9b86c1d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.525705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.531759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.533086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.533580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.534517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] PATH_INFO: `/v3/projects/65ed875cce384b159628663bfeeec71e/users/3dacaf144a4f47aaa9974cc729f68961/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.540760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 930/1849] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:34.544821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.557220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a19514-c89c-41e3-b34a-c6aeca4735f1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.561160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ed875cce384b159628663bfeeec71e, user_id=3dacaf144a4f47aaa9974cc729f68961, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.562802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.563629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.564381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.564745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.564935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.566229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.567944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.588278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1850] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:34 2022] PUT /identity/v3/projects/65ed875cce384b159628663bfeeec71e/users/3dacaf144a4f47aaa9974cc729f68961/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:34.594524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.616430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 931/1851] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:34.620563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.714724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8518a1-1e10-49b9-8fb4-30df4aa6c074 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.743252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433c0b2-cd85-497b-a150-e5131ce2d8dd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.752763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.753374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.753670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.753940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.754643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.756606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.789538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 932/1852] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/users => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:34.791759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.803536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.804417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.804997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.806065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.807412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.810363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.813285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb292c-8556-457b-9a43-4657aa566c10 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.821971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1853] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:34.822610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.823301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.823598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.823891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.824742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.826110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.826527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.844384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 933/1854] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:34.845621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dc7c0-d806-402a-a5a2-d1d29bffad23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.848243 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.850765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f07be-bc85-45aa-b351-121b99681d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.851813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f07be-bc85-45aa-b351-121b99681d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.852103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f07be-bc85-45aa-b351-121b99681d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.852411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f07be-bc85-45aa-b351-121b99681d22 admin admin] PATH_INFO: `/v3/projects/65ed875cce384b159628663bfeeec71e/users/3dacaf144a4f47aaa9974cc729f68961/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.854469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f07be-bc85-45aa-b351-121b99681d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ed875cce384b159628663bfeeec71e, user_id=3dacaf144a4f47aaa9974cc729f68961, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.857157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f07be-bc85-45aa-b351-121b99681d22 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.857744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35345de-1743-4b11-aee5-2fc6ff254db5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.863748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.864838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.865127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.865412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.866437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.868046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.868926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1855] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:34 2022] PUT /identity/v3/projects/65ed875cce384b159628663bfeeec71e/users/3dacaf144a4f47aaa9974cc729f68961/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:34.879480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.879919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.880318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.884129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 934/1856] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:34.885092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:34.902278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad2f0-c056-4d3f-bc46-a440f8c1509d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:34.910358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] MFA Rules not processed for user `3dacaf144a4f47aaa9974cc729f68961`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:34.911631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:34.924821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:34.954072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:34.971798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea4ce2-9e02-4fbb-a586-69415ab5fd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:34.972891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4ce2-9e02-4fbb-a586-69415ab5fd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:34.973247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4ce2-9e02-4fbb-a586-69415ab5fd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:34.973557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4ce2-9e02-4fbb-a586-69415ab5fd14 admin admin] PATH_INFO: `/v3/projects/773dc8a1b7da4001a347c232d408920b/users/8a032de375864cd3a614b90b9fe303dd/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:34.989063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4ce2-9e02-4fbb-a586-69415ab5fd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773dc8a1b7da4001a347c232d408920b, user_id=8a032de375864cd3a614b90b9fe303dd, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:34.991763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4ce2-9e02-4fbb-a586-69415ab5fd14 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:34.992990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1857] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:35.006106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 935/1858] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:34 2022] PUT /identity/v3/projects/773dc8a1b7da4001a347c232d408920b/users/8a032de375864cd3a614b90b9fe303dd/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:35.006569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.016133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.016468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.016818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.042935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] MFA Rules not processed for user `8a032de375864cd3a614b90b9fe303dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:35.044208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:35.064707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae776f-81b4-4e9c-8dbd-3ede8341b6a0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.111947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 936/1859] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:35.116592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.117736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.118389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.118714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.118945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.187197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.187439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b656bb-ae8f-44a8-8892-274acb1e24ab None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.189218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.193152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.193822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.194105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.194389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.195084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.197033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.199877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1860] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:35.228147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 937/1861] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:35 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:35.232068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.243734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9068199-8261-41ac-a1ca-43601db62c8b service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.259152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:35.310637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.311311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.311894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.312296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.313001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.315506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.341188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1862] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:35 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:35.346124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.357013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30db195-6c63-4a5b-a630-cde705594b1a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.363005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.363675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.364057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.364377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.365109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.366863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.381121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 938/1863] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:35.385645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.397641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389d70-3651-42d4-b037-8784e4361c0e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.402384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.403276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.403600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.404230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] PATH_INFO: `/v3/projects/488c10c4997a4756a8e3be415552bfb1/users/068ba3c15dbc4a31a9692b4c7319ed03/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.421713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=488c10c4997a4756a8e3be415552bfb1, user_id=068ba3c15dbc4a31a9692b4c7319ed03, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.425384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.442380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1864] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:35 2022] PUT /identity/v3/projects/488c10c4997a4756a8e3be415552bfb1/users/068ba3c15dbc4a31a9692b4c7319ed03/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:35.446249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.457343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799df1f-875b-41c7-8648-c4ed12349416 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.487201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:35.534553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.535236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.535525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.535805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.536571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.538239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.549784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 939/1865] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:35.554148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.565665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0ba8b-a6ae-43ff-ad4a-735aeabbed7b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.572978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.579857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cd367-0bcb-40bc-bd7c-5598fbb01150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.581798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cd367-0bcb-40bc-bd7c-5598fbb01150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.582286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cd367-0bcb-40bc-bd7c-5598fbb01150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.582621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cd367-0bcb-40bc-bd7c-5598fbb01150 admin admin] PATH_INFO: `/v3/projects/488c10c4997a4756a8e3be415552bfb1/users/068ba3c15dbc4a31a9692b4c7319ed03/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.584864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cd367-0bcb-40bc-bd7c-5598fbb01150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488c10c4997a4756a8e3be415552bfb1, user_id=068ba3c15dbc4a31a9692b4c7319ed03, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.587642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cd367-0bcb-40bc-bd7c-5598fbb01150 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.600750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1866] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:35 2022] PUT /identity/v3/projects/488c10c4997a4756a8e3be415552bfb1/users/068ba3c15dbc4a31a9692b4c7319ed03/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:35.612994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.613621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.613913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.646765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] MFA Rules not processed for user `068ba3c15dbc4a31a9692b4c7319ed03`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:35.648021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:35.662036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949e3af-76a7-4287-b113-94ab4551a677 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.737754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dacaf144a4f47aaa9974cc729f68961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65ed875cce384b159628663bfeeec71e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.738538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.738830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.739250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.740298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:35.740832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.743704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.749768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1867] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:35.766256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 940/1868] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:35.770769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.781742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.784912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:35.789452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56200fba-25f2-48bf-a9e4-ad869df8e650 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.798533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dacaf144a4f47aaa9974cc729f68961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65ed875cce384b159628663bfeeec71e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.799717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.800222 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.800708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.801987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.805820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.852680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1869] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:35 2022] POST /identity/v3/projects => generated 354 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:35.854262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:35.860992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b21b97-f3d8-4436-a723-a3de995dd29c tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.889694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:35.898950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:35.927392 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.927999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.928316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.928659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.933368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bdf25-453a-4767-9de0-04ed050ead84 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:35.941908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dacaf144a4f47aaa9974cc729f68961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65ed875cce384b159628663bfeeec71e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:35.942550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:35.942853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:35.943147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:35.943998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:35.944576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:35.946364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:35.963139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1870] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:31:36.005597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:36.009563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:36.019080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 941/1871] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:36.080552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:36.092850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9534e1f-59c6-4125-9f47-f891a0a98aa4 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:36.101250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3dacaf144a4f47aaa9974cc729f68961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65ed875cce384b159628663bfeeec71e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:36.102231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:36.102618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:36.105170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] PATH_INFO: `/v3/projects/e2f946022d4f4aa78fbd625452b3a9f4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:36.106147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e2f946022d4f4aa78fbd625452b3a9f4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:36.110237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:36.169857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f946022d4f4aa78fbd625452b3a9f4', 'request_id': 'req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.180972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:36.182799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f946022d4f4aa78fbd625452b3a9f4', 'request_id': 'req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.191946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f946022d4f4aa78fbd625452b3a9f4', 'request_id': 'req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.192881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81821de-d4fc-4c1e-a081-14e9156fc896 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:36.206136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] The token cache is being invalidate because project e2f946022d4f4aa78fbd625452b3a9f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:36.206538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.207399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.209298 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1872] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:36 2022] DELETE /identity/v3/projects/e2f946022d4f4aa78fbd625452b3a9f4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:36.248072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:36.248851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:36.249481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:36.249759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:36.252257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:36.316565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:36.324890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2267c-340d-467f-9ddf-0de2b1623331 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:36.331910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 942/1873] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:36.699830 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:36.701293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:36.775576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f028e-8d02-4b7a-b728-f9a8c44c7866 tempest-VolumeQuotaClassesTest-1358036071 tempest-VolumeQuotaClassesTest-1358036071-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:36.780367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:36.781232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:36.781512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:36.781785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] PATH_INFO: `/v3/users/d4e22bb199fd4fd9b799ee13c4942460` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:36.783614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e22bb199fd4fd9b799ee13c4942460)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:36.785745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:36.837564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e22bb199fd4fd9b799ee13c4942460'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.838227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e22bb199fd4fd9b799ee13c4942460'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.841933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:36.845652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e22bb199fd4fd9b799ee13c4942460'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.854861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e22bb199fd4fd9b799ee13c4942460'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.867235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e22bb199fd4fd9b799ee13c4942460'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:36.873174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1874] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:36 2022] DELETE /identity/v3/users/d4e22bb199fd4fd9b799ee13c4942460 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:36.878490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:36.878998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:36.879116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:36.896225 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:36.897439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:36.919782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:36.959680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 943/1875] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:36.969371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:36.970782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:37.026708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8bf03-7228-482b-a61c-16102620cdb3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:37.030180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:37.030816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:37.031100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:37.031379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:37.040482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:37.041900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:37.051697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1876] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:37.316354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:37.317812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:37.329921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c85cf2-05ea-4885-be64-3b5578d418b2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:37.334599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:37.335471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:37.335761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:37.336042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] PATH_INFO: `/v3/projects/37ba4bda52d14d20b5fc94d3cc6c644f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:37.337281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ba4bda52d14d20b5fc94d3cc6c644f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:37.339237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:37.386729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ba4bda52d14d20b5fc94d3cc6c644f', 'request_id': 'req-64cc0417-ec22-46c4-ae70-143e090074a1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.393788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ba4bda52d14d20b5fc94d3cc6c644f', 'request_id': 'req-64cc0417-ec22-46c4-ae70-143e090074a1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.400252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ba4bda52d14d20b5fc94d3cc6c644f', 'request_id': 'req-64cc0417-ec22-46c4-ae70-143e090074a1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.410962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] The token cache is being invalidate because project 37ba4bda52d14d20b5fc94d3cc6c644f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:37.411249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.411882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.413456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 944/1877] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:37 2022] DELETE /identity/v3/projects/37ba4bda52d14d20b5fc94d3cc6c644f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:37.417427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:37.426993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fbbe9-f9af-499a-8de1-a99d3e0c9f05 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:37.492862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:37.493781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:37.494069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:37.494351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] PATH_INFO: `/v3/users/3dacaf144a4f47aaa9974cc729f68961` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:37.495567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dacaf144a4f47aaa9974cc729f68961)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:37.497691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:37.545201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dacaf144a4f47aaa9974cc729f68961'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.545854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dacaf144a4f47aaa9974cc729f68961'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.554584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dacaf144a4f47aaa9974cc729f68961'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.562599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dacaf144a4f47aaa9974cc729f68961'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.572084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:37.577914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dacaf144a4f47aaa9974cc729f68961'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:37.585317 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1878] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:37 2022] DELETE /identity/v3/users/3dacaf144a4f47aaa9974cc729f68961 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:37.981550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:37.983089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:37.983527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:37.986068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.002685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc0417-ec22-46c4-ae70-143e090074a1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.087425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cee21-4686-41b4-8549-308962423cb7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.093335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.095942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.096458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.097030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] PATH_INFO: `/v3/users/b486b8536d1045cfab9d3ebb4c51df54` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.098859 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b486b8536d1045cfab9d3ebb4c51df54)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.101407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.103979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.104467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.104753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.105390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] PATH_INFO: `/v3/projects/65ed875cce384b159628663bfeeec71e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.106825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ed875cce384b159628663bfeeec71e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.109873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.161535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b486b8536d1045cfab9d3ebb4c51df54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.162240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b486b8536d1045cfab9d3ebb4c51df54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.171196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b486b8536d1045cfab9d3ebb4c51df54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.171626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ed875cce384b159628663bfeeec71e', 'request_id': 'req-d8dc003c-88e1-4a60-b224-59a3bb198e99'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.175043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:38.180428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b486b8536d1045cfab9d3ebb4c51df54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.182140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ed875cce384b159628663bfeeec71e', 'request_id': 'req-d8dc003c-88e1-4a60-b224-59a3bb198e99'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.192089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ed875cce384b159628663bfeeec71e', 'request_id': 'req-d8dc003c-88e1-4a60-b224-59a3bb198e99'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.195223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6c80c74d-3428-4c7e-8425-60ae97e37ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b486b8536d1045cfab9d3ebb4c51df54'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.201671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1879] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:37 2022] DELETE /identity/v3/users/b486b8536d1045cfab9d3ebb4c51df54 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:38.206648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b69ec-7df6-4502-92db-9675a0f2fe22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.206771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] The token cache is being invalidate because project 65ed875cce384b159628663bfeeec71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:38.206994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b69ec-7df6-4502-92db-9675a0f2fe22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.207118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.207278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b69ec-7df6-4502-92db-9675a0f2fe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.207754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8dc003c-88e1-4a60-b224-59a3bb198e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.209419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 945/1880] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:37 2022] DELETE /identity/v3/projects/65ed875cce384b159628663bfeeec71e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:38.213863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.214152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.214431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.230091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-033b69ec-7df6-4502-92db-9675a0f2fe22 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:38.231349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-033b69ec-7df6-4502-92db-9675a0f2fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.232244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:38.233398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.290683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1881] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:38.291034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 946/1882] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:38.299952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.300278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.300562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.301126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.302600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.320440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:38.321628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.329526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1883] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:38.333508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.338674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:38.342872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55ea1-bdd2-4e4d-8a17-81e3bc471b6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.349667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.350559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.350850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.351610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.352669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:38.352978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.354594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.361969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5140d48-1254-4a00-9651-8314aadc9f7b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.365681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.366432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.366719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.366963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1884] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:31:38.367181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.371199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.377217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.378657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.381524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c683f-5006-4985-a3f8-ac2b4b84c474 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.386304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.386944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.387198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.387484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.388168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.388915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 947/1885] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:38.390159 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.420880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1886] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:38.425787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.435862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c72e7-5202-4401-bf7f-65c18d7d80dc service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.503892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.504760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.505241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.505659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.506702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.509089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.529007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 948/1887] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:38.533391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.544654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d0ce7-29a8-495d-b5e0-c32a5a8f794c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.550441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.551109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.551395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.551672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.552601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.554271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.567705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1888] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:38.569162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.580391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010d662-95be-4d38-af81-15ac5022fbb0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.585238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a20f8-48c8-4c2b-8731-b9d9b826b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.585903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a20f8-48c8-4c2b-8731-b9d9b826b5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.586185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a20f8-48c8-4c2b-8731-b9d9b826b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.586459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a20f8-48c8-4c2b-8731-b9d9b826b5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.587184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a20f8-48c8-4c2b-8731-b9d9b826b5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.588828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a20f8-48c8-4c2b-8731-b9d9b826b5a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.599635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 949/1889] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:38.608879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.618261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bcf76-afd3-4a09-a805-63d30c4779fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.622941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.623826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.624500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.624688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] PATH_INFO: `/v3/projects/d7a403993b2b41f0b1807a5ab2e1f19e/users/ebca8b1c31f14accad7de26fe7464922/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.639997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7a403993b2b41f0b1807a5ab2e1f19e, user_id=ebca8b1c31f14accad7de26fe7464922, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.643321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.658243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1890] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:38 2022] PUT /identity/v3/projects/d7a403993b2b41f0b1807a5ab2e1f19e/users/ebca8b1c31f14accad7de26fe7464922/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:38.671761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.672298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.672691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.680473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:38.694002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.695933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.714169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] MFA Rules not processed for user `ebca8b1c31f14accad7de26fe7464922`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:38.715537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.788707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77c3b8-f36d-4ccf-9acf-c058fc574710 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.794064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.795085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.795348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.795610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] PATH_INFO: `/v3/projects/29bf653d38d64a8c80a02160bf1f3fd6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.797072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29bf653d38d64a8c80a02160bf1f3fd6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.799196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.806573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 950/1891] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:38.810732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.822513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06476ff-5132-44f1-93a9-54a318d1f17b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.827731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.828430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.828984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.829852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.830693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.832493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.854034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29bf653d38d64a8c80a02160bf1f3fd6', 'request_id': 'req-cf7403b3-7381-414e-93d1-fb3acdb19099'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.860461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29bf653d38d64a8c80a02160bf1f3fd6', 'request_id': 'req-cf7403b3-7381-414e-93d1-fb3acdb19099'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.866699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 951/1892] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:38.867724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.871883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29bf653d38d64a8c80a02160bf1f3fd6', 'request_id': 'req-cf7403b3-7381-414e-93d1-fb3acdb19099'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.884576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] The token cache is being invalidate because project 29bf653d38d64a8c80a02160bf1f3fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:38.884868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.885538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:38.887164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1893] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:38 2022] DELETE /identity/v3/projects/29bf653d38d64a8c80a02160bf1f3fd6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:38.890979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.893173 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.922751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:38.951242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ff845-0fff-4d3c-a38c-3ace628a3a3b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.956431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.957101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.957385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.957664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.958357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.960301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:38.979326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7403b3-7381-414e-93d1-fb3acdb19099 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:38.986919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 952/1894] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:38 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:38.989264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:38.990464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:38.991100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:38.991672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:38.992237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:38.992551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] PATH_INFO: `/v3/users/7ec13db808b14c9ab72cc6d7381d5678` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:38.994192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec13db808b14c9ab72cc6d7381d5678)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:38.996611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.009911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586964d0-7266-43ad-99aa-d26a2d91cd94 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.025505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.026654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.026925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.027367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.028466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.030903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.046836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 953/1895] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:39.049029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.072653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec13db808b14c9ab72cc6d7381d5678'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.073708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec13db808b14c9ab72cc6d7381d5678'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.073842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde59f9-3a97-44e3-ad2a-685320643218 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.081111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.081810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.082436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.082597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.083379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.084027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec13db808b14c9ab72cc6d7381d5678'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.085032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.094141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec13db808b14c9ab72cc6d7381d5678'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.098051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 954/1896] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:39.102180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.103989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:39.112968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-12f2460b-6898-4d59-8c71-76e6e8f523fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec13db808b14c9ab72cc6d7381d5678'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.117760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a464-3ecc-49b7-9e9d-5fd630cdde0f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.119114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1897] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:38 2022] DELETE /identity/v3/users/7ec13db808b14c9ab72cc6d7381d5678 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:39.155660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:39.190315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.191277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.191565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.191916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] PATH_INFO: `/v3/projects/e8e4bb9ac30b4559910fefc7f2e91a98/users/907bdfaf1aaf492ebf8e761ab6750d02/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.210452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e4bb9ac30b4559910fefc7f2e91a98, user_id=907bdfaf1aaf492ebf8e761ab6750d02, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.213474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.227000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 955/1898] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:39 2022] PUT /identity/v3/projects/e8e4bb9ac30b4559910fefc7f2e91a98/users/907bdfaf1aaf492ebf8e761ab6750d02/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:39.238846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.239226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.239515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.266137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] MFA Rules not processed for user `907bdfaf1aaf492ebf8e761ab6750d02`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:39.267980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:39.339252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1899] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:39.343806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.410641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fbde7-24d7-43d0-89e5-54c141d071c1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.415626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.416436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.416754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.417989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.418488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.419363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:39.422644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.425420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.433397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a3848-ac07-4a9b-948f-7ff34a516f12 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.439184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.440209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.440465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.440850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] PATH_INFO: `/v3/projects/e508aa73352348a8a1360121bca35abd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.442061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e508aa73352348a8a1360121bca35abd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.444079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.460226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 956/1900] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:39.461500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.479059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:39.496960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e508aa73352348a8a1360121bca35abd', 'request_id': 'req-4a3e1496-a25d-442a-b9bb-695d7115442f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.503558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e508aa73352348a8a1360121bca35abd', 'request_id': 'req-4a3e1496-a25d-442a-b9bb-695d7115442f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.510912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e508aa73352348a8a1360121bca35abd', 'request_id': 'req-4a3e1496-a25d-442a-b9bb-695d7115442f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.523011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] The token cache is being invalidate because project e508aa73352348a8a1360121bca35abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:39.523309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.523968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4a3e1496-a25d-442a-b9bb-695d7115442f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:39.525690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1901] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:39 2022] DELETE /identity/v3/projects/e508aa73352348a8a1360121bca35abd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:39.530254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.530580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.530865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.531539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a64d4-8215-446a-97ac-53d6085befb8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.548486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:39.549690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:39.578437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:39.589068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5f8ac-e4db-4d44-a152-93e041acdff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.589679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5f8ac-e4db-4d44-a152-93e041acdff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.589972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5f8ac-e4db-4d44-a152-93e041acdff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.590252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5f8ac-e4db-4d44-a152-93e041acdff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.590951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5f8ac-e4db-4d44-a152-93e041acdff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.592971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5f8ac-e4db-4d44-a152-93e041acdff0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.612597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1902] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:39.613712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 957/1903] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:39.620383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.620818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.621134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.621413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.621733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:39.634360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c2356-9ff6-4fc3-b89c-36e1b9a60350 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.639635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.640337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.640620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.640876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:39.640987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.641699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.642173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:39.643302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.650057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 958/1904] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:39.654084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.656070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1905] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:39.658734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.664468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11ac77-c358-4e51-ad7e-0bfacc81fdd5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.669055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.669474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a7a98-2aad-4199-9ec4-c82352825ac6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.669718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.669994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.670274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.670991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:39.671293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.673128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.674162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.675021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.675308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.675589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] PATH_INFO: `/v3/projects/687ae1f508d44a758b62dab6ad8ee1ac/users/987220fe455f40bd98ae799296ede4da/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.684960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 959/1906] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:31:39.689734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.692153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=687ae1f508d44a758b62dab6ad8ee1ac, user_id=987220fe455f40bd98ae799296ede4da, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.694869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.699186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2714fe1-95cd-40fc-86c7-7e8f86bffb0b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.704020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.704824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.705108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.705388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.706079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.707982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.708698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1907] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:39 2022] PUT /identity/v3/projects/687ae1f508d44a758b62dab6ad8ee1ac/users/987220fe455f40bd98ae799296ede4da/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:39.712459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.734371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 960/1908] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:39.738456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.774536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f0fe3-5f71-46d1-a33c-fbd03685c91f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.802570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2c5fd-57fa-462a-b7a4-496f2ffb4b15 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.803825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:39.807570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.808299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.808564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.808850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.809662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.811604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.836164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 961/1909] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:39.836369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.836946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.837229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.837354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.837524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.838234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.839881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.846855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4474cb5-ac86-43ac-8267-c3ec269e2049 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.854637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1910] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:39.854889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.855461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.855750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.856043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.856784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.858387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.861682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.872050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 962/1911] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:39.872414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cd95e-e018-47c9-a193-f6f778faa321 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.873568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.879892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5a973-d221-4aac-aa21-526e6e4661de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.881218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a973-d221-4aac-aa21-526e6e4661de admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.881957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a973-d221-4aac-aa21-526e6e4661de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.881957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5a973-d221-4aac-aa21-526e6e4661de admin admin] PATH_INFO: `/v3/projects/687ae1f508d44a758b62dab6ad8ee1ac/users/987220fe455f40bd98ae799296ede4da/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.883768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5a973-d221-4aac-aa21-526e6e4661de admin admin] RBAC: Authorizing `identity:create_grant(project_id=687ae1f508d44a758b62dab6ad8ee1ac, user_id=987220fe455f40bd98ae799296ede4da, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.885464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad78cc5-9b1d-4c49-9903-a32b580e9e0c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.886498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5a973-d221-4aac-aa21-526e6e4661de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.890964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:39.891756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.891986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.892455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.893502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:39.895402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:39.897383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1912] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:39 2022] PUT /identity/v3/projects/687ae1f508d44a758b62dab6ad8ee1ac/users/987220fe455f40bd98ae799296ede4da/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:39.907616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:39.907899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:39.908211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:39.914334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 963/1913] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:39.915455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:39.925896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30edff0-1558-4e12-8be6-4dfff0407efc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:39.952803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] MFA Rules not processed for user `987220fe455f40bd98ae799296ede4da`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:39.954545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:40.006403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e812a5-afb9-490f-8506-e47a63a3b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.007420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e812a5-afb9-490f-8506-e47a63a3b50b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.007705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e812a5-afb9-490f-8506-e47a63a3b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.007980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e812a5-afb9-490f-8506-e47a63a3b50b admin admin] PATH_INFO: `/v3/projects/e1dfecdff7ae40c98d3ffd3359dc19f9/users/be5152bf45284c308153b0e06b9b88e8/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.028147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:40.028147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e812a5-afb9-490f-8506-e47a63a3b50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dfecdff7ae40c98d3ffd3359dc19f9, user_id=be5152bf45284c308153b0e06b9b88e8, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.028394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e812a5-afb9-490f-8506-e47a63a3b50b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.035839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1914] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:40.043311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 964/1915] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:39 2022] PUT /identity/v3/projects/e1dfecdff7ae40c98d3ffd3359dc19f9/users/be5152bf45284c308153b0e06b9b88e8/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:40.046915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.049060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:40.053814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.054106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.054389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.084921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] MFA Rules not processed for user `be5152bf45284c308153b0e06b9b88e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:40.086014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:40.122109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ce516-88c4-4ebd-a748-2d57871981b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.162988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 965/1916] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:40.167229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.186191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.186849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.187140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.187419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.195177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:40.247581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9d0b5-4644-43d6-b8e0-bec0e61c3c22 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.257120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.257810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.258092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.258475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.259074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.262058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.272133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.273796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.285887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1917] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:40.305259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 966/1918] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:40 2022] POST /identity/v3/projects => generated 391 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:40.306581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.317444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53562f4-d34b-4c7b-a49c-0859f8502982 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.377966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.378642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.378929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.379209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.379897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.381892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.407466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1919] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:40 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:40.408749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.419005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345a377-e36f-4776-a8a3-07ee9ed327ef admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.424324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.425425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.425944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.426533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.427163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.429125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.441375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 967/1920] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:40.445670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.456547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f05c24-04e9-439f-85b2-1d54c2c2c90c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.461209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8004585-f768-4887-8a42-349123294024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.462085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004585-f768-4887-8a42-349123294024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.462372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004585-f768-4887-8a42-349123294024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.462653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8004585-f768-4887-8a42-349123294024 admin admin] PATH_INFO: `/v3/projects/cb15a5b6c8c14630bc2aac0c27b192a5/users/5a64c11d6f8243268ab56998d18307c8/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.480558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8004585-f768-4887-8a42-349123294024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb15a5b6c8c14630bc2aac0c27b192a5, user_id=5a64c11d6f8243268ab56998d18307c8, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.483246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8004585-f768-4887-8a42-349123294024 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.500399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1921] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:40 2022] PUT /identity/v3/projects/cb15a5b6c8c14630bc2aac0c27b192a5/users/5a64c11d6f8243268ab56998d18307c8/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:40.504436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.514512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b80c-a9e2-4db8-972d-405d90e1ddef admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.570174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:40.579203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a98490-efba-4dca-aa34-a003a46b6712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.580068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a98490-efba-4dca-aa34-a003a46b6712 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.580473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a98490-efba-4dca-aa34-a003a46b6712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.580671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a98490-efba-4dca-aa34-a003a46b6712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.581403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a98490-efba-4dca-aa34-a003a46b6712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.583563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a98490-efba-4dca-aa34-a003a46b6712 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.596124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 968/1922] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:40.600080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8004585-f768-4887-8a42-349123294024 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.611946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8004585-f768-4887-8a42-349123294024 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.616860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.617687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.617969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.618269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] PATH_INFO: `/v3/projects/cb15a5b6c8c14630bc2aac0c27b192a5/users/5a64c11d6f8243268ab56998d18307c8/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.620365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb15a5b6c8c14630bc2aac0c27b192a5, user_id=5a64c11d6f8243268ab56998d18307c8, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.622924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.634359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1923] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:40 2022] PUT /identity/v3/projects/cb15a5b6c8c14630bc2aac0c27b192a5/users/5a64c11d6f8243268ab56998d18307c8/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:40.645806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.646167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.646471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.676060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] MFA Rules not processed for user `5a64c11d6f8243268ab56998d18307c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:40.677497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:40.754995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 969/1924] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:40.767050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.769209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:40.834059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcafa6-0fe3-4c7c-85fc-346e938bb9a0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.838153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.838848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.839355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.839909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.852957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.854477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.858739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:40.865168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1925] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:40.893804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.903641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd89b1a-8e84-47a0-9cbc-f12e43c2140c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.907495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.908142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.908474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.908754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.910572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:40.918198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:40.926772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d47e99-65bd-4e5d-851a-5a472011716c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:40.930885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bc4c8-5cfe-4023-a3db-46dd065ea0ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:40.931513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc4c8-5cfe-4023-a3db-46dd065ea0ac service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:40.931796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc4c8-5cfe-4023-a3db-46dd065ea0ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:40.932071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc4c8-5cfe-4023-a3db-46dd065ea0ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:40.972749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.974235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac82584-ad72-4b90-bf92-fdc63b043647 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:40.983386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 970/1926] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:40.992287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bc4c8-5cfe-4023-a3db-46dd065ea0ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:40.993732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bc4c8-5cfe-4023-a3db-46dd065ea0ac service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:41.003658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1927] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:47.002602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97562360-eb26-4a02-bf44-7c2bc21778ce None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:47.003636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97562360-eb26-4a02-bf44-7c2bc21778ce None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:47.004303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97562360-eb26-4a02-bf44-7c2bc21778ce None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:47.006616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 971/1928] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:31:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:31:47.018034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:47.018585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:47.018585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:47.056431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:47.065217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:47.158910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1929] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:47.201877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97562360-eb26-4a02-bf44-7c2bc21778ce None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:47.205120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-97562360-eb26-4a02-bf44-7c2bc21778ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:47.241554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97562360-eb26-4a02-bf44-7c2bc21778ce None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:47.250831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18741887-29ea-4e15-b3d8-68ca93f8a01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:47.252236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18741887-29ea-4e15-b3d8-68ca93f8a01f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:47.252371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18741887-29ea-4e15-b3d8-68ca93f8a01f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:47.252634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18741887-29ea-4e15-b3d8-68ca93f8a01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:47.263534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18741887-29ea-4e15-b3d8-68ca93f8a01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:47.265169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18741887-29ea-4e15-b3d8-68ca93f8a01f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:47.278096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 972/1930] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:48.105957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:48.107489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:48.173374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aabc2d-e683-47f2-a125-44dc0fb13cfd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:48.177762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:48.178628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:48.178911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:48.179183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] PATH_INFO: `/v3/users/1e41fa711df7482a8c2d682eeb1557b2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:48.180438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e41fa711df7482a8c2d682eeb1557b2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:48.182338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:48.217543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:48.238111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e41fa711df7482a8c2d682eeb1557b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.239020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e41fa711df7482a8c2d682eeb1557b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.247232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e41fa711df7482a8c2d682eeb1557b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.255280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e41fa711df7482a8c2d682eeb1557b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.269541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e41fa711df7482a8c2d682eeb1557b2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.276878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1931] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:48 2022] DELETE /identity/v3/users/1e41fa711df7482a8c2d682eeb1557b2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:48.281448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:48.282032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:48.282719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:48.315164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:48.316660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:48.386646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 973/1932] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:48.397096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:48.398714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:48.464098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cedce7-061e-4c21-8181-61eb30af92eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:48.468364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:48.469041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:48.469336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:48.470140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:48.481094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:48.482632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:48.495405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1933] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:48.818522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:48.819962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:48.827054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:48.834166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a4374-2707-4701-843e-cb8d5d109695 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:48.839348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:48.840401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:48.840799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:48.841084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] PATH_INFO: `/v3/projects/219ebde4c8294d928da98b603d4ecb05` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:48.842290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=219ebde4c8294d928da98b603d4ecb05)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:48.844234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:48.892062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '219ebde4c8294d928da98b603d4ecb05', 'request_id': 'req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.900287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '219ebde4c8294d928da98b603d4ecb05', 'request_id': 'req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.906538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '219ebde4c8294d928da98b603d4ecb05', 'request_id': 'req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.917531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] The token cache is being invalidate because project 219ebde4c8294d928da98b603d4ecb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:48.917815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.918431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:48.919979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 974/1934] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:48 2022] DELETE /identity/v3/projects/219ebde4c8294d928da98b603d4ecb05 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:48.924001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:48.933838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d86ef-6f37-4716-a5c5-f7b5e21f291c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:48.989066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:48.989938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:48.990228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:48.990513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] PATH_INFO: `/v3/users/57d018da8d8f49bb8216bd3340a6d9d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:48.991684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d018da8d8f49bb8216bd3340a6d9d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:48.993685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.040248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d018da8d8f49bb8216bd3340a6d9d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.041067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d018da8d8f49bb8216bd3340a6d9d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.044674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:49.049836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d018da8d8f49bb8216bd3340a6d9d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.059229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d018da8d8f49bb8216bd3340a6d9d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.073246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f35e013-8f77-4931-888f-64719d87966e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d018da8d8f49bb8216bd3340a6d9d5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.079925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1935] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:48 2022] DELETE /identity/v3/users/57d018da8d8f49bb8216bd3340a6d9d5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:49.352675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:49.356530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:49.370764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca339b7e-0cbc-48c7-bf65-b36f136fcb70 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:49.450469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:49.451451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.451976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.452459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] PATH_INFO: `/v3/projects/5d9beebcdc3b4e08ad90eb55adde32ba` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.454404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9beebcdc3b4e08ad90eb55adde32ba)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:49.456699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.536751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9beebcdc3b4e08ad90eb55adde32ba', 'request_id': 'req-9f7510f4-798e-445f-9e16-3799acb33067'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.546032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9beebcdc3b4e08ad90eb55adde32ba', 'request_id': 'req-9f7510f4-798e-445f-9e16-3799acb33067'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.558367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9beebcdc3b4e08ad90eb55adde32ba', 'request_id': 'req-9f7510f4-798e-445f-9e16-3799acb33067'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.570050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] The token cache is being invalidate because project 5d9beebcdc3b4e08ad90eb55adde32ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:49.570341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.571006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9f7510f4-798e-445f-9e16-3799acb33067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:49.572745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 975/1936] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:49 2022] DELETE /identity/v3/projects/5d9beebcdc3b4e08ad90eb55adde32ba => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:49.577811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.578257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.578457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.596961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:49.598322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:49.689184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1937] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:49.696826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.697116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.697399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.719020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:49.720161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:49.735560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 976/1938] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:49.736843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:49.747208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52917b21-ed8d-4d63-8921-5f8ddc1a2b16 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:49.753931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:49.754612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.754902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.755187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.755923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:49.756270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:49.757906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.774937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1939] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:49.776702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:49.786641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa4d8d-5f40-44ba-8279-f1e98e380e4c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:49.791250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:49.791911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.792237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.792523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.793230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:49.795171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.800418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:49.824344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 977/1940] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:49 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:49.828609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:49.838468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c2388-0700-447b-b5eb-62d499650c1c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:49.895167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:49.895841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.896129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.896444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.897135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:49.900274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.922304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1941] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:49 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:49.923766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:49.936129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0816779-6e3e-4ec7-b101-8c6f1ec70d82 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:49.939165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:49.939819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.940104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.940417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.941136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:49.942760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.952498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 978/1942] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:49.956968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:49.966969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc7c0-5930-4112-a77c-352105ed2e70 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:49.971710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed40df-994d-42ad-8d80-f80a1e7a74b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:49.972560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed40df-994d-42ad-8d80-f80a1e7a74b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:49.972844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed40df-994d-42ad-8d80-f80a1e7a74b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:49.973155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed40df-994d-42ad-8d80-f80a1e7a74b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:49.973942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed40df-994d-42ad-8d80-f80a1e7a74b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:49.975650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed40df-994d-42ad-8d80-f80a1e7a74b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:49.985745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1943] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:49.989985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.000206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c81f3a-5ddf-4ac8-99c5-c7806e81ccc0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.005372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.006242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.006526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.006804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] PATH_INFO: `/v3/projects/4d9abd6dcd6a439f9b47e2392c5fe0e7/users/50fcd4761f9449d3b4f2127d44bac653/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.021714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d9abd6dcd6a439f9b47e2392c5fe0e7, user_id=50fcd4761f9449d3b4f2127d44bac653, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.024473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.036939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 979/1944] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:49 2022] PUT /identity/v3/projects/4d9abd6dcd6a439f9b47e2392c5fe0e7/users/50fcd4761f9449d3b4f2127d44bac653/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:50.047120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.047482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.047773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.074731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] MFA Rules not processed for user `50fcd4761f9449d3b4f2127d44bac653`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:50.075941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:50.101155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:50.143597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1945] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:50.151748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.214994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4bd73-bb7b-4c7e-8a71-e10365784939 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.220280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.220953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.221261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.221538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.222304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.224318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.249055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 980/1946] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:50 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:50.253612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.264325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d570a5-5fc7-48ef-87dc-0616cf2b2d83 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.319884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.320541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.320831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.321115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.321821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.323770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.343498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1947] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:50 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:50.347675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.358035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce517e5-1f49-42ef-a4ce-76e5c7db3cab admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.363802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.364559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.364851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.365133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.365865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.367465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.379543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 981/1948] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:50.383795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.394070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c969e-4453-4c31-8518-6d6c10d3e4b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.398634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.399618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.399902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.400202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] PATH_INFO: `/v3/projects/5df13c7497274c37886ef415414b4b8a/users/414ff1024f4749be86e23248c4c80286/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.415946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df13c7497274c37886ef415414b4b8a, user_id=414ff1024f4749be86e23248c4c80286, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.418638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.433271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1949] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:50 2022] PUT /identity/v3/projects/5df13c7497274c37886ef415414b4b8a/users/414ff1024f4749be86e23248c4c80286/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:50.437308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.448152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d87bd-43cf-4e36-afd0-ce38ff05b8e5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.511766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3c877-8eee-435d-a175-757dd90c432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.512771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c877-8eee-435d-a175-757dd90c432b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.516037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c877-8eee-435d-a175-757dd90c432b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.516037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3c877-8eee-435d-a175-757dd90c432b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.516037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3c877-8eee-435d-a175-757dd90c432b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.516037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3c877-8eee-435d-a175-757dd90c432b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.528875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 982/1950] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:50.534001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.545558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cca675-6d3d-443a-93ea-bd31a4635ba5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.550358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.551297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.551677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.552054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] PATH_INFO: `/v3/projects/5df13c7497274c37886ef415414b4b8a/users/414ff1024f4749be86e23248c4c80286/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.554812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df13c7497274c37886ef415414b4b8a, user_id=414ff1024f4749be86e23248c4c80286, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.557513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.570548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1951] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:50 2022] PUT /identity/v3/projects/5df13c7497274c37886ef415414b4b8a/users/414ff1024f4749be86e23248c4c80286/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:50.583702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.584294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.584811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.626793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] MFA Rules not processed for user `414ff1024f4749be86e23248c4c80286`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:50.628506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:50.640324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:50.736682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 983/1952] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:50.752506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:50.759328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:50.870720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f19897-d73e-4d7b-a52e-e47a4c2c54f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:50.880288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cdd60-23b1-452e-82e5-3cdc6163fd4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:50.881514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cdd60-23b1-452e-82e5-3cdc6163fd4c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:50.881808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cdd60-23b1-452e-82e5-3cdc6163fd4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:50.882178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cdd60-23b1-452e-82e5-3cdc6163fd4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:50.901595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cdd60-23b1-452e-82e5-3cdc6163fd4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:50.905457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cdd60-23b1-452e-82e5-3cdc6163fd4c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:50.925559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1953] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:51.414082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:51.415567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:51.484079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3598f-33a3-4ac8-955c-4ec52652b2f4 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:51.488943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:51.490087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:51.490539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:51.490822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] PATH_INFO: `/v3/users/be5152bf45284c308153b0e06b9b88e8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:51.492477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5152bf45284c308153b0e06b9b88e8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:51.496035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:51.562203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5152bf45284c308153b0e06b9b88e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:51.570246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5152bf45284c308153b0e06b9b88e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:51.570972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5152bf45284c308153b0e06b9b88e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:51.583598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5152bf45284c308153b0e06b9b88e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:51.590970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5152bf45284c308153b0e06b9b88e8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:51.598442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 984/1954] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:51 2022] DELETE /identity/v3/users/be5152bf45284c308153b0e06b9b88e8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:51.603008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:51.603344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:51.603642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:51.629169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:51.630406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:51.642973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:51.697194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1955] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:51.706865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:51.708522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:51.742054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:51.775154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709d608-6852-44db-8eb2-c601cd938e2a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:51.778879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:51.779532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:51.779820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:51.780118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:51.800748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:51.802765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:51.814711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 985/1956] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:52.230953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:52.232660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:52.246591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab51f7-6f96-4859-af9b-4e680832394d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:52.253045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:52.253983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:52.254346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:52.254634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] PATH_INFO: `/v3/projects/e1dfecdff7ae40c98d3ffd3359dc19f9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:52.256360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1dfecdff7ae40c98d3ffd3359dc19f9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:52.258364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:52.352777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1dfecdff7ae40c98d3ffd3359dc19f9', 'request_id': 'req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.359708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1dfecdff7ae40c98d3ffd3359dc19f9', 'request_id': 'req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.382350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1dfecdff7ae40c98d3ffd3359dc19f9', 'request_id': 'req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.396482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] The token cache is being invalidate because project e1dfecdff7ae40c98d3ffd3359dc19f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:52.396775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.397927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.400164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1957] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:52 2022] DELETE /identity/v3/projects/e1dfecdff7ae40c98d3ffd3359dc19f9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:52.404336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:52.426142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00d315-e1b9-446a-a157-decb2bc2e7b8 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:52.497731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:52.498712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:52.499020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:52.499607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] PATH_INFO: `/v3/users/5a64c11d6f8243268ab56998d18307c8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:52.500907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a64c11d6f8243268ab56998d18307c8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:52.503164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:52.558278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a64c11d6f8243268ab56998d18307c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.570768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a64c11d6f8243268ab56998d18307c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.572782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a64c11d6f8243268ab56998d18307c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.601258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a64c11d6f8243268ab56998d18307c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.619308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-06616583-3f94-4c48-a8dc-f285b93ae0e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a64c11d6f8243268ab56998d18307c8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.632236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 986/1958] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:52 2022] DELETE /identity/v3/users/5a64c11d6f8243268ab56998d18307c8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:52.673850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:52.675415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:52.692718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d524cd7-e3d6-4d87-aae6-2efe7615cf07 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:52.756840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:52.757810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:52.758163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:52.758514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] PATH_INFO: `/v3/users/50fcd4761f9449d3b4f2127d44bac653` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:52.759794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50fcd4761f9449d3b4f2127d44bac653)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:52.761828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:52.827209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:52.829684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50fcd4761f9449d3b4f2127d44bac653'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.830522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50fcd4761f9449d3b4f2127d44bac653'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.839909 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50fcd4761f9449d3b4f2127d44bac653'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.849433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50fcd4761f9449d3b4f2127d44bac653'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.862079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50fcd4761f9449d3b4f2127d44bac653'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:52.868881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1959] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:52 2022] DELETE /identity/v3/users/50fcd4761f9449d3b4f2127d44bac653 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:52.873242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:52.873536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:52.873822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:52.898387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:52.900556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:52.936906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:52.939238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:52.961833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:52.980976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 987/1960] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:52.994475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:52.996137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:53.040052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd7373-47fa-41b8-a1cc-9d19d8f86fb1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:53.048441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:53.053314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.053629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.053923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] PATH_INFO: `/v3/projects/cb15a5b6c8c14630bc2aac0c27b192a5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.057661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb15a5b6c8c14630bc2aac0c27b192a5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:53.060540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:53.132883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097c51b-5569-4da1-9ce9-19550464c26a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:53.150335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb15a5b6c8c14630bc2aac0c27b192a5', 'request_id': 'req-6ad33352-8291-414e-92d4-6b712c766afd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.165920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb15a5b6c8c14630bc2aac0c27b192a5', 'request_id': 'req-6ad33352-8291-414e-92d4-6b712c766afd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.180241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb15a5b6c8c14630bc2aac0c27b192a5', 'request_id': 'req-6ad33352-8291-414e-92d4-6b712c766afd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.201900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] The token cache is being invalidate because project cb15a5b6c8c14630bc2aac0c27b192a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:53.202268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.203609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ad33352-8291-414e-92d4-6b712c766afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.209615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:53.210487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1961] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:52 2022] DELETE /identity/v3/projects/cb15a5b6c8c14630bc2aac0c27b192a5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:53.210944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.211400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.211828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.215536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:53.216767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.217129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.217424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.277599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:53.288006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:53.391887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:53.401070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340cae39-5d01-4d83-857a-248f05dd7249 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:53.430935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 988/1962] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:31:53.455783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:53.488808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1963] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:53.490957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.491526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.491948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.512575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:53.514049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:53.527754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 989/1964] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:53.528214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:53.555580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae4c9c-e245-4247-a230-87950fdd49e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:53.568350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:53.569463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.569908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.570345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.571249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:53.571726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:53.576437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:53.621750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1965] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:31:53.628203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:53.642477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dee6a-c2a3-43ef-b971-79f7f3641a79 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:53.648748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:53.651224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:53.666191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:53.666886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.668953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.668953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.668953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:53.677692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e977e-c1ad-44d9-a32a-7347fb3fb85f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:53.683875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:53.689764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:53.692926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:53.693634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:53.696012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] PATH_INFO: `/v3/users/8a032de375864cd3a614b90b9fe303dd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:53.698320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a032de375864cd3a614b90b9fe303dd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:53.709071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:53.768601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 990/1966] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:53 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:31:53.773202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:53.828513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a032de375864cd3a614b90b9fe303dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.829861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a032de375864cd3a614b90b9fe303dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.840508 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a032de375864cd3a614b90b9fe303dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.853558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:53.854367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a032de375864cd3a614b90b9fe303dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.881473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a032de375864cd3a614b90b9fe303dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:53.902588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1967] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:53 2022] DELETE /identity/v3/users/8a032de375864cd3a614b90b9fe303dd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:53.902588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:53.902858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:53.912212 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f6812-53f4-4af9-8056-ee50c9bef5c6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.014690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa14e4-5d92-4d34-b109-72eb01440dc2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.019364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.020034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.020361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.020652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.023061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.024080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c010-114c-47d6-903c-a0a72fc2daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.025472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c010-114c-47d6-903c-a0a72fc2daa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.025862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c010-114c-47d6-903c-a0a72fc2daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.026197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c010-114c-47d6-903c-a0a72fc2daa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.026911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c010-114c-47d6-903c-a0a72fc2daa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.029872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c010-114c-47d6-903c-a0a72fc2daa5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.061853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 991/1968] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:53 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:54.065537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.065933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.066219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.088385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:54.089925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.099688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 992/1969] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:54.100488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.102156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.105766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.107225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.117674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1970] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:54.118463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.120627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.123978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a473254-107b-42ec-88e5-e39a15780f9e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.131233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.132194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.132453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.132785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] PATH_INFO: `/v3/projects/4d9abd6dcd6a439f9b47e2392c5fe0e7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.134044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d9abd6dcd6a439f9b47e2392c5fe0e7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.135938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.136391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d8224-0a8b-41c7-af87-ec1d734a561e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.142198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.143084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.143576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.144037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.145100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.147039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.148910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:54.162523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1971] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:54.163709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.165637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.206129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d9abd6dcd6a439f9b47e2392c5fe0e7', 'request_id': 'req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.222259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d9abd6dcd6a439f9b47e2392c5fe0e7', 'request_id': 'req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.232647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d9abd6dcd6a439f9b47e2392c5fe0e7', 'request_id': 'req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.244272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] The token cache is being invalidate because project 4d9abd6dcd6a439f9b47e2392c5fe0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:54.244568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.245418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.250645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 993/1972] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:54 2022] DELETE /identity/v3/projects/4d9abd6dcd6a439f9b47e2392c5fe0e7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:54.251505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.252554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea251b-8927-48f0-94bf-4fd13009ffde admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.253025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.321782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:54.327900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.330097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.330681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.331309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.334236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.348795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5aab93-653e-4ad8-8e56-0dd8f7df5bbd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.349090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.352797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.354462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.355238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.355630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.356138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.357125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.358841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.366479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1973] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:54.367438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.370611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.370952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 994/1974] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:54.376401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.388977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6b8a9-6532-4ee7-a49b-b9003b532c5e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.390584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b44bb-689f-4d34-89dc-56de801819bb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.395199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214a288-a400-4cd2-a0ac-09fa938581c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.396159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214a288-a400-4cd2-a0ac-09fa938581c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.396488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214a288-a400-4cd2-a0ac-09fa938581c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.396800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214a288-a400-4cd2-a0ac-09fa938581c4 admin admin] PATH_INFO: `/v3/projects/766c7c6678a24b6f9c2c06a3b8699bba/users/6ba52875fd1d43c49e3bd24b2cd46394/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.397685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.398835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.399093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.399365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] PATH_INFO: `/v3/users/414ff1024f4749be86e23248c4c80286` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.401231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414ff1024f4749be86e23248c4c80286)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.403468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.416495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214a288-a400-4cd2-a0ac-09fa938581c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=766c7c6678a24b6f9c2c06a3b8699bba, user_id=6ba52875fd1d43c49e3bd24b2cd46394, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.420026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214a288-a400-4cd2-a0ac-09fa938581c4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.440946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 995/1975] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:54 2022] PUT /identity/v3/projects/766c7c6678a24b6f9c2c06a3b8699bba/users/6ba52875fd1d43c49e3bd24b2cd46394/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:54.451997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.453034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.453624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.468580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414ff1024f4749be86e23248c4c80286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.469495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414ff1024f4749be86e23248c4c80286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.478701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414ff1024f4749be86e23248c4c80286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.488484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414ff1024f4749be86e23248c4c80286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.493540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] MFA Rules not processed for user `6ba52875fd1d43c49e3bd24b2cd46394`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:54.495577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.504970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414ff1024f4749be86e23248c4c80286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:54.512990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1976] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:54 2022] DELETE /identity/v3/users/414ff1024f4749be86e23248c4c80286 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:54.625679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 996/1977] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:54.637762 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.639186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.670639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:54.700085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932be5a-cb98-46f7-b7e0-d73f0c7a8a59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.703977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:54.704661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:54.704950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:54.705295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:54.793434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:54.794901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:54.812642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1978] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:54.828220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.830335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.889116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:54.893926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:54.983039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcf04b-e5fb-4cd7-bfc6-bd7bb4cc0c64 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:54.997207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b076bfc-4736-4ee4-8b16-40db1cd165e6 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.004861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.004861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.004861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.005197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] PATH_INFO: `/v3/projects/773dc8a1b7da4001a347c232d408920b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.008313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773dc8a1b7da4001a347c232d408920b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.010316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.039479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:55.067162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.068471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.068764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.069050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] PATH_INFO: `/v3/users/6ba52875fd1d43c49e3bd24b2cd46394` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.070276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ba52875fd1d43c49e3bd24b2cd46394)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.072267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.075942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773dc8a1b7da4001a347c232d408920b', 'request_id': 'req-81283f49-59ff-4b0f-8029-846c76b510d9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.083042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773dc8a1b7da4001a347c232d408920b', 'request_id': 'req-81283f49-59ff-4b0f-8029-846c76b510d9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.093796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773dc8a1b7da4001a347c232d408920b', 'request_id': 'req-81283f49-59ff-4b0f-8029-846c76b510d9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.107390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] The token cache is being invalidate because project 773dc8a1b7da4001a347c232d408920b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:55.107580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.108241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.110195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1979] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:54 2022] DELETE /identity/v3/projects/773dc8a1b7da4001a347c232d408920b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:55.111045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.114454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.132730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:55.160097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ba52875fd1d43c49e3bd24b2cd46394'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.174399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ba52875fd1d43c49e3bd24b2cd46394'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.175295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ba52875fd1d43c49e3bd24b2cd46394'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.197838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ba52875fd1d43c49e3bd24b2cd46394'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.206244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ba52875fd1d43c49e3bd24b2cd46394'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.206721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81283f49-59ff-4b0f-8029-846c76b510d9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.211713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.212751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 997/1980] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:54 2022] DELETE /identity/v3/users/6ba52875fd1d43c49e3bd24b2cd46394 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:55.212866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.213326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.213687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] PATH_INFO: `/v3/projects/5df13c7497274c37886ef415414b4b8a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.214988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df13c7497274c37886ef415414b4b8a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.216295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.217474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.217696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.271671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df13c7497274c37886ef415414b4b8a', 'request_id': 'req-276098db-bd9e-4d93-a945-b2162e681962'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.277689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df13c7497274c37886ef415414b4b8a', 'request_id': 'req-276098db-bd9e-4d93-a945-b2162e681962'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.285928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df13c7497274c37886ef415414b4b8a', 'request_id': 'req-276098db-bd9e-4d93-a945-b2162e681962'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.289567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99fa5e-43b5-4b93-b498-5c83b41b4877 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.297853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] The token cache is being invalidate because project 5df13c7497274c37886ef415414b4b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:55.298149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.298815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-276098db-bd9e-4d93-a945-b2162e681962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.300499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1981] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:55 2022] DELETE /identity/v3/projects/5df13c7497274c37886ef415414b4b8a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:55.301992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce466f-0b86-4c65-92b8-f93ce89d3ff7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.302273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce466f-0b86-4c65-92b8-f93ce89d3ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.302556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce466f-0b86-4c65-92b8-f93ce89d3ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.320761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fdce466f-0b86-4c65-92b8-f93ce89d3ff7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:55.322045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fdce466f-0b86-4c65-92b8-f93ce89d3ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.348815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.349873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.350086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.350368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] PATH_INFO: `/v3/users/068ba3c15dbc4a31a9692b4c7319ed03` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.351601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] RBAC: Authorizing `identity:delete_user(user_id=068ba3c15dbc4a31a9692b4c7319ed03)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.353649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.375062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:55.388906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1982] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:55.390622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.391133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.392109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.414884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:55.416100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.425194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1983] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:55.426051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.427374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.436920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068ba3c15dbc4a31a9692b4c7319ed03'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.446793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068ba3c15dbc4a31a9692b4c7319ed03'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.447584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068ba3c15dbc4a31a9692b4c7319ed03'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.460455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068ba3c15dbc4a31a9692b4c7319ed03'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.467505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-92008699-c1a6-4036-804f-7151a9fd765c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068ba3c15dbc4a31a9692b4c7319ed03'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.472650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:55.476802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 998/1984] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:55 2022] DELETE /identity/v3/users/068ba3c15dbc4a31a9692b4c7319ed03 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:55.476802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.476802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.476802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.496076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:55.496367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3cc8a-8067-49b5-a847-1e582477d9e6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.497359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.558132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.558999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.559359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.559648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.561637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.574192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 999/1985] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:55.578826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.591654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a8c68-ea7e-4a4c-a172-47a40dc25496 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.597071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.597881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.598257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.598588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.599395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:55.599978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.601456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.620098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1000/1986] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:55.624952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.636119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145431e7-1029-4ac2-a4dd-13f3943be336 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.643155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.644715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.650214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.650923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.651969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.652299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.653010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.655569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.657121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1987] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:55.693042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1001/1988] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:55 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:55.697570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.710500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6210c-8342-43a3-ac2c-8470a6a479b7 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.786684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.791358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.791358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.791358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.791358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.792451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.794112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.794277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.823778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1989] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:55 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:55.827871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.832044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e60be8-f356-401d-bf78-15c8dca335f2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.839571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0970d-998e-4bcb-8cb7-3cd8680e35d1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.845563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.847000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.847299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.848379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.848481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.849077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.849192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.849401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.849746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] PATH_INFO: `/v3/projects/488c10c4997a4756a8e3be415552bfb1` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.850886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=488c10c4997a4756a8e3be415552bfb1)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.851076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.852835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.868350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1990] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:55.870088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.882277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05d39a-1b83-4eae-870a-5acbca184d19 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.886953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.887613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.887903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.888207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.889111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:55.891159 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:55.903526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1991] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:55.908258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:55.914112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488c10c4997a4756a8e3be415552bfb1', 'request_id': 'req-f117f3ad-58eb-4142-a723-92be1a488080'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.919121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:55.920732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f274a1-8a85-429d-bf57-36c74fbbcef6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:55.925237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488c10c4997a4756a8e3be415552bfb1', 'request_id': 'req-f117f3ad-58eb-4142-a723-92be1a488080'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.930970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:55.934294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488c10c4997a4756a8e3be415552bfb1', 'request_id': 'req-f117f3ad-58eb-4142-a723-92be1a488080'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.948649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] The token cache is being invalidate because project 488c10c4997a4756a8e3be415552bfb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:55.949065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.949612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f117f3ad-58eb-4142-a723-92be1a488080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:55.951361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1002/1992] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:55 2022] DELETE /identity/v3/projects/488c10c4997a4756a8e3be415552bfb1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:55.957928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614bd3-6965-4984-9e23-073801be7353 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.957928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614bd3-6965-4984-9e23-073801be7353 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.957928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614bd3-6965-4984-9e23-073801be7353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:55.977002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4f614bd3-6965-4984-9e23-073801be7353 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:55.978381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4f614bd3-6965-4984-9e23-073801be7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:55.993844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:55.994731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:55.995041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:55.995339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] PATH_INFO: `/v3/projects/afc83e5f26bc4db7a6c7fc4b223d1518/users/f3f829e06fa44c4d88bd65474444c5de/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.011229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc83e5f26bc4db7a6c7fc4b223d1518, user_id=f3f829e06fa44c4d88bd65474444c5de, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.014109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.028613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/1993] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:55 2022] PUT /identity/v3/projects/afc83e5f26bc4db7a6c7fc4b223d1518/users/f3f829e06fa44c4d88bd65474444c5de/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:31:56.029641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.033961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:56.043565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1003/1994] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:31:56.045576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768eccc1-02e2-475d-a5c5-f5c546b39728 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.045869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768eccc1-02e2-475d-a5c5-f5c546b39728 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.046148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768eccc1-02e2-475d-a5c5-f5c546b39728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.078208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-768eccc1-02e2-475d-a5c5-f5c546b39728 None None] MFA Rules not processed for user `f3f829e06fa44c4d88bd65474444c5de`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:56.079946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-768eccc1-02e2-475d-a5c5-f5c546b39728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:56.111529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76aa5-ce43-4460-8894-4c87d9e4c149 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.117429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.119185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.119185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.119185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] PATH_INFO: `/v3/projects/766c7c6678a24b6f9c2c06a3b8699bba` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.120057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=766c7c6678a24b6f9c2c06a3b8699bba)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.122013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.172160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1004/1995] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:56.174217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031438d-81fa-43bf-a52e-b6e275a9ed22 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.174518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031438d-81fa-43bf-a52e-b6e275a9ed22 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.174806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031438d-81fa-43bf-a52e-b6e275a9ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.196099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '766c7c6678a24b6f9c2c06a3b8699bba', 'request_id': 'req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:56.198015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4031438d-81fa-43bf-a52e-b6e275a9ed22 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:56.208448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4031438d-81fa-43bf-a52e-b6e275a9ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:56.225216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '766c7c6678a24b6f9c2c06a3b8699bba', 'request_id': 'req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:56.240361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '766c7c6678a24b6f9c2c06a3b8699bba', 'request_id': 'req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:56.262885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] The token cache is being invalidate because project 766c7c6678a24b6f9c2c06a3b8699bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:56.263181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:56.264125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:56.267506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/1996] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:56 2022] DELETE /identity/v3/projects/766c7c6678a24b6f9c2c06a3b8699bba => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:56.275481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.280264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:56.320859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1005/1997] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:56.322437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.322798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.323383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.327672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:56.362119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:56.363387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:56.379867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1006/1998] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:56.379867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.384443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f3705-c089-437c-a454-747bd98f25e7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.392575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.393930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.394254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.394626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.395788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:56.396360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.396906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce0ed6-0293-4a85-ad27-04659ab7cab9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.398310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217fc6-4814-4637-b779-fc49a748d14f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.401997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:56.408090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.408960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.410431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.410721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.414528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.417080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.434114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1007/1999] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:56.436016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.436785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.437134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.464043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:56.465666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:56.466904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/2000] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:56.468445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.587137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1008/2001] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:56.591423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.603696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed532f17-af03-405c-942c-cc88efb252c0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.605127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc2f41-24d2-4fb6-8f87-1bee64116ee7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.613720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.614458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.614790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.615076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.615831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.618030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.625889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.626679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.627863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.628238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.629002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.631495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.662432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1009/2002] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/users => generated 289 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:56.666944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.677561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/2003] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:56.678898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.681248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a114ba-d916-48e4-a38d-40727b84cf9d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.765938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.768459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.770298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.770627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.773456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:31:56.773786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.775460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cabf89-757b-4472-a184-08521f4709b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.775725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.783409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.784578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.785030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.785829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.786713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.789595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.804570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1010/2004] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:31:56.809724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/2005] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:56.809862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.810592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.822896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c1abb-be16-4263-a67f-d25ee9cfca5f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.824546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4319fbcb-8317-41ed-b892-340214d47776 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.830641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.831401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.831755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.832099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.832908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.835102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.835233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.835787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.836080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.836387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.837084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.838992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:56.867485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1011/2006] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/users => generated 294 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:56.870242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.879712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/2007] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:56 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:56.884045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:56.888143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b851b9-633c-423a-8a33-3fbdf6eca4a2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:56.894629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:56.963306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:56.980637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:56.981414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:56.981776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:56.982197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:56.983255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:56.985928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.004726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940ad11-0305-4835-a0c0-52f71d200fad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.009001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1012/2008] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.010015 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.011560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.013656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.014100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.014842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.020238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.022084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.025328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d71ef-bb37-4989-b134-c52f652e32de admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.031903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.032888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.033409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.033892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.037102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/2009] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.037102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.038579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.041031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.045126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa8ce8-238c-4dc0-9038-819eb07cfd46 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.051310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.052408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.052736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.053060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] PATH_INFO: `/v3/projects/f20f12cbb1854376b67d5b39fb2c86c3/users/eb202ec3c6ad43b5aabeba01810b236f/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.067443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1013/2010] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/users => generated 286 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:57.069500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.077234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20f12cbb1854376b67d5b39fb2c86c3, user_id=eb202ec3c6ad43b5aabeba01810b236f, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.080982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.087476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0787-2308-4997-90a0-57aa86cb8482 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.093477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84f67a-33e1-4d07-b81b-d5f742884262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.095051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84f67a-33e1-4d07-b81b-d5f742884262 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.095797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84f67a-33e1-4d07-b81b-d5f742884262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.096619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84f67a-33e1-4d07-b81b-d5f742884262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.098108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84f67a-33e1-4d07-b81b-d5f742884262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.100540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84f67a-33e1-4d07-b81b-d5f742884262 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.110902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/2011] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:57 2022] PUT /identity/v3/projects/f20f12cbb1854376b67d5b39fb2c86c3/users/eb202ec3c6ad43b5aabeba01810b236f/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:57.112017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.124689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1014/2012] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.127017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.127460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.127893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.163686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] MFA Rules not processed for user `eb202ec3c6ad43b5aabeba01810b236f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:57.164960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:57.184437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf5110-2545-4e53-8942-ed5a189582b5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.190337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.191035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.191323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.191603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.192362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.193986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.205090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/2013] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.205728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.215730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c3cdd-1687-4e84-95f4-dac3d32063f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.221300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.222175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.222462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.222740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] PATH_INFO: `/v3/projects/6c6eedf0820b411aaf0715a1240750db/users/37a77db6c3dd492fa2fc9d7e5b72725e/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.240075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1015/2014] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:57.240075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.240336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6eedf0820b411aaf0715a1240750db, user_id=37a77db6c3dd492fa2fc9d7e5b72725e, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.240696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.248621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ada76-4e51-4d0c-a0b4-0bcc5b389293 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.254297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/2015] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:57 2022] PUT /identity/v3/projects/6c6eedf0820b411aaf0715a1240750db/users/37a77db6c3dd492fa2fc9d7e5b72725e/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:57.257244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.305933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c60a2-4093-4f13-9d21-ce56fbbd4872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.306639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c60a2-4093-4f13-9d21-ce56fbbd4872 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.306926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c60a2-4093-4f13-9d21-ce56fbbd4872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.307292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c60a2-4093-4f13-9d21-ce56fbbd4872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.308028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c60a2-4093-4f13-9d21-ce56fbbd4872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.308991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:57.309737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c60a2-4093-4f13-9d21-ce56fbbd4872 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.323812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1016/2016] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.324001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f14be-6eea-4388-8af9-8296cbf3cfc2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.324100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.324100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.324100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.327307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.327959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.328266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.329065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.329747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.331648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.349679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] MFA Rules not processed for user `37a77db6c3dd492fa2fc9d7e5b72725e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:57.350991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:57.361533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/2017] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:57.361533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.386652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:57.424263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe340f-14ef-4161-81d3-c9a435c6d85f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.426116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1017/2018] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:57.427286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.433067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.433978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.434262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.434541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] PATH_INFO: `/v3/projects/134292b9fbb94480a7bcaa7509d1e283/users/5e1db2b5cde545c993d6b8a5c5458d3a/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.439733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aabe16e-d6a4-4356-9e52-a947c08d5a34 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.445446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.446124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.446407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.446688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.447385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.449368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.452579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134292b9fbb94480a7bcaa7509d1e283, user_id=5e1db2b5cde545c993d6b8a5c5458d3a, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.455877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.469885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1018/2019] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:57.470413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/2020] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:57 2022] PUT /identity/v3/projects/134292b9fbb94480a7bcaa7509d1e283/users/5e1db2b5cde545c993d6b8a5c5458d3a/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:57.470761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.481851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.482017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70afb22-bdd0-4df7-8a11-4849b390ffef admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.536436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d7659-6791-4436-a2ad-113a0fd18162 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.538290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.538960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.539252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.539540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.540257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.541459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017d40d-dc21-4e2a-9b65-f02185366604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.542128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d40d-dc21-4e2a-9b65-f02185366604 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.542224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.542419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d40d-dc21-4e2a-9b65-f02185366604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.542701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d40d-dc21-4e2a-9b65-f02185366604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.543432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d40d-dc21-4e2a-9b65-f02185366604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.545089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d40d-dc21-4e2a-9b65-f02185366604 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.555533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2021] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.560160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.560491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.560799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.568773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1019/2022] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:57.569601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.591216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] MFA Rules not processed for user `5e1db2b5cde545c993d6b8a5c5458d3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:57.593510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:57.653668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:57.670059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5855a62-c1e2-4651-8f50-13ca6bccc1e8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.674897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.675796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.676081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.676567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] PATH_INFO: `/v3/projects/29a0be97e73a46ddbaf4f2321b010bc0/users/266eed857b774741a00a16e3a470b2bd/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.687696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2023] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:57.687696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.692943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a0be97e73a46ddbaf4f2321b010bc0, user_id=266eed857b774741a00a16e3a470b2bd, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.696613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.707853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2a3dc-53a2-4def-b82c-fc378142f952 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.713425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1020/2024] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:57 2022] PUT /identity/v3/projects/29a0be97e73a46ddbaf4f2321b010bc0/users/266eed857b774741a00a16e3a470b2bd/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:57.714584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.716519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:57.760138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:57.770565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.772092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.772460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.772764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.774585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.776676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.800870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2025] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:31:57.803432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.815943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884130e0-e023-4229-bb55-6287693f400a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.818317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7663c-ecc2-478b-b396-b2ec93bc2fd1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.825853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.826524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.827548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.828332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.829149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.829428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.830170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.831548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.831879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.832758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.844033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2026] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.848077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.862019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8dac-d741-477e-b1f1-dbf942008e6a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.866934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.867647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.867938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.868368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.869092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.870794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.884256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2027] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:31:57.884910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.897230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e37e5-736e-4d99-99ae-f0d79fd54fdf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.903237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee1f17-f628-4ee9-9d40-1263e48c3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:57.904128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee1f17-f628-4ee9-9d40-1263e48c3657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.904590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee1f17-f628-4ee9-9d40-1263e48c3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.904992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee1f17-f628-4ee9-9d40-1263e48c3657 admin admin] PATH_INFO: `/v3/projects/29a0be97e73a46ddbaf4f2321b010bc0/users/266eed857b774741a00a16e3a470b2bd/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.907227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee1f17-f628-4ee9-9d40-1263e48c3657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a0be97e73a46ddbaf4f2321b010bc0, user_id=266eed857b774741a00a16e3a470b2bd, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.909199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:57.910648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.910956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee1f17-f628-4ee9-9d40-1263e48c3657 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:57.920529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1021/2028] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:57.921333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:57.925528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2029] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:57 2022] PUT /identity/v3/projects/29a0be97e73a46ddbaf4f2321b010bc0/users/266eed857b774741a00a16e3a470b2bd/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:57.937488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:57.937787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:57.938185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:57.939281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0a376-93d7-454b-bebd-5a42bdfe3fb2 service glance] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:57.975680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] MFA Rules not processed for user `266eed857b774741a00a16e3a470b2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:57.977562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:58.021885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:58.022897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.023242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.023513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] PATH_INFO: `/v3/projects/89319fc7c8014c2c81dbcef296ccbfc1/users/7518b53c83d04688b828c3004f1dbd70/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.045186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89319fc7c8014c2c81dbcef296ccbfc1, user_id=7518b53c83d04688b828c3004f1dbd70, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:58.049395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:58.072870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1022/2030] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:57 2022] PUT /identity/v3/projects/89319fc7c8014c2c81dbcef296ccbfc1/users/7518b53c83d04688b828c3004f1dbd70/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:58.077048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:58.081651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2031] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:58.136879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:58.157202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987b708-d5ee-4230-8978-d18dd16d97d4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:58.162312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:58.164053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.164399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.164678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.165417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:58.167233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:58.181707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1023/2032] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:31:58.182278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:58.196911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb87459-441d-4c34-a142-0b242ab4293d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:58.203385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92ce3f-616b-44ff-8c27-e69539b33a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:58.204577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ce3f-616b-44ff-8c27-e69539b33a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.204860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ce3f-616b-44ff-8c27-e69539b33a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.205140 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92ce3f-616b-44ff-8c27-e69539b33a62 admin admin] PATH_INFO: `/v3/projects/89319fc7c8014c2c81dbcef296ccbfc1/users/7518b53c83d04688b828c3004f1dbd70/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.207982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ce3f-616b-44ff-8c27-e69539b33a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89319fc7c8014c2c81dbcef296ccbfc1, user_id=7518b53c83d04688b828c3004f1dbd70, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:58.211972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92ce3f-616b-44ff-8c27-e69539b33a62 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:58.227092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2033] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:31:58 2022] PUT /identity/v3/projects/89319fc7c8014c2c81dbcef296ccbfc1/users/7518b53c83d04688b828c3004f1dbd70/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:58.234297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:58.237833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:58.244018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.244576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.244736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.269860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:58.280709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] MFA Rules not processed for user `7518b53c83d04688b828c3004f1dbd70`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:58.283273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:58.315531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b7cca6-1cc6-4f3e-80ad-a614df0ed06a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:58.361743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2034] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:58.374631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:58.375345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.375628 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.375906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.377734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:58.443151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:58.444764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:58.454546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1024/2035] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:58.519980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:58.521460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:58.536422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f2b63-1b9b-4629-814e-b229dca23e97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:58.540229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301222d-1c5b-4062-bf83-3c5be2a744e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:58.541063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301222d-1c5b-4062-bf83-3c5be2a744e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.541360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301222d-1c5b-4062-bf83-3c5be2a744e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.541641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301222d-1c5b-4062-bf83-3c5be2a744e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.551037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301222d-1c5b-4062-bf83-3c5be2a744e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:58.552498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301222d-1c5b-4062-bf83-3c5be2a744e7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:58.562174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2036] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:58.666818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:58.723112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:58.744641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1942-9daa-41f9-8b79-3fae406ddd40 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:58.749765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:58.750772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.751137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.751531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] PATH_INFO: `/v3/users/37a77db6c3dd492fa2fc9d7e5b72725e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.752837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a77db6c3dd492fa2fc9d7e5b72725e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:58.754819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:58.807451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a77db6c3dd492fa2fc9d7e5b72725e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:58.814800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a77db6c3dd492fa2fc9d7e5b72725e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:58.815529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a77db6c3dd492fa2fc9d7e5b72725e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:58.829085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a77db6c3dd492fa2fc9d7e5b72725e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:58.836082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a77db6c3dd492fa2fc9d7e5b72725e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:58.842128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1025/2037] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:58 2022] DELETE /identity/v3/users/37a77db6c3dd492fa2fc9d7e5b72725e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:58.847311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:58.847413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:58.847652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:58.866059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:31:58.867341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:58.928411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2038] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:31:58.938491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:58.939901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:59.013487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1bb88-09cd-4385-8abc-3e58a58f9bce admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:59.017795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:59.018450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:59.018737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:59.019018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:59.028328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:59.029779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:59.039376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1026/2039] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:31:59.295553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:59.297013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:59.309411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ba5dc-6f02-4402-aa62-4c620f606afd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:59.314494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:59.315408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:59.315693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:59.316106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] PATH_INFO: `/v3/projects/6c6eedf0820b411aaf0715a1240750db` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:59.317657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6eedf0820b411aaf0715a1240750db)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:59.319708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:59.366574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6eedf0820b411aaf0715a1240750db', 'request_id': 'req-4dec20cc-7a01-47bc-8627-b6e531d808c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.372584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6eedf0820b411aaf0715a1240750db', 'request_id': 'req-4dec20cc-7a01-47bc-8627-b6e531d808c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.380199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6eedf0820b411aaf0715a1240750db', 'request_id': 'req-4dec20cc-7a01-47bc-8627-b6e531d808c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.392827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] The token cache is being invalidate because project 6c6eedf0820b411aaf0715a1240750db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:31:59.393114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.393849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.395396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2040] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:59 2022] DELETE /identity/v3/projects/6c6eedf0820b411aaf0715a1240750db => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:59.399491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:59.409872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489bf86-82bf-4e62-8a57-39b6f4375f60 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:59.465646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:59.466522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:59.466804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:59.467078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] PATH_INFO: `/v3/users/7518b53c83d04688b828c3004f1dbd70` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:59.468289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7518b53c83d04688b828c3004f1dbd70)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:59.471974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:59.491074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:59.527347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7518b53c83d04688b828c3004f1dbd70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.536413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7518b53c83d04688b828c3004f1dbd70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.537422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7518b53c83d04688b828c3004f1dbd70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.550700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7518b53c83d04688b828c3004f1dbd70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.558257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2cdebdcf-92e1-410b-a07f-df85153dedc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7518b53c83d04688b828c3004f1dbd70'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.570552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1027/2041] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:31:59 2022] DELETE /identity/v3/users/7518b53c83d04688b828c3004f1dbd70 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:31:59.855693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:31:59.857149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:31:59.871316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec20cc-7a01-47bc-8627-b6e531d808c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:31:59.876200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:31:59.936125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:31:59.937105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:31:59.937453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:31:59.937731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] PATH_INFO: `/v3/projects/89319fc7c8014c2c81dbcef296ccbfc1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:31:59.939541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=89319fc7c8014c2c81dbcef296ccbfc1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:31:59.941488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:31:59.991974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89319fc7c8014c2c81dbcef296ccbfc1', 'request_id': 'req-b2a37fe1-7c5b-4294-8806-4678f02c1cff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:31:59.998014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89319fc7c8014c2c81dbcef296ccbfc1', 'request_id': 'req-b2a37fe1-7c5b-4294-8806-4678f02c1cff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:00.016752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89319fc7c8014c2c81dbcef296ccbfc1', 'request_id': 'req-b2a37fe1-7c5b-4294-8806-4678f02c1cff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:00.024819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] The token cache is being invalidate because project 89319fc7c8014c2c81dbcef296ccbfc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:00.025111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:00.025767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a37fe1-7c5b-4294-8806-4678f02c1cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:00.027318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2042] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:31:59 2022] DELETE /identity/v3/projects/89319fc7c8014c2c81dbcef296ccbfc1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:00.035811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.038577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.038874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.061416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:00.062770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:00.126930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1028/2043] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:00.134945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.135322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.135671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.155782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:00.156976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:00.165267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2044] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:00.169577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.180251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea4e22-c424-42e0-b653-f6d8f22e244b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.184857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.185515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.185794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.186079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.186808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:00.187114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.188851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.204022 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1029/2045] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:00.206610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.216518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad7387-b5fc-41f0-b841-b1ec014c7152 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.222634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.223297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.223582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.223862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.224605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.226530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.253788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2046] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:00.258005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.269437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d60cf6-03f1-48c6-a71d-c4b3ed2be72a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.342081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.342773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.343116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.343436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.344747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.346790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.370425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1030/2047] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:00.371629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.382851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6d5b2-c282-4475-b9cb-8921501aa0bb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.387730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.388426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.388719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.389003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.389746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.391950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.405257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2048] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:00.409248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.422486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835962f-3f7c-4e13-ab5e-7aa178789285 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.431610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d5c4d-08ea-4758-8adb-8ec792b324f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.432314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5c4d-08ea-4758-8adb-8ec792b324f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.432599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5c4d-08ea-4758-8adb-8ec792b324f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.432874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d5c4d-08ea-4758-8adb-8ec792b324f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.433602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d5c4d-08ea-4758-8adb-8ec792b324f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.436096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d5c4d-08ea-4758-8adb-8ec792b324f2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.439503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:00.451365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1031/2049] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:00.456350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.467003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491fd2f-1c15-4c5d-9323-2d73bfb9b702 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.471645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.472576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.472860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.473342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] PATH_INFO: `/v3/projects/f41c49a69e3f474598c1ebae85293551/users/a40cfdd7ddce4cd6b313ff889cf032d2/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.489375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41c49a69e3f474598c1ebae85293551, user_id=a40cfdd7ddce4cd6b313ff889cf032d2, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.492005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.505290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2050] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:00 2022] PUT /identity/v3/projects/f41c49a69e3f474598c1ebae85293551/users/a40cfdd7ddce4cd6b313ff889cf032d2/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:00.517132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.517420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.517840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.546704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] MFA Rules not processed for user `a40cfdd7ddce4cd6b313ff889cf032d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:00.547910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:00.620518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1032/2051] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:00.625888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.661471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:00.698911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6450c-d681-474b-bb97-c1c22b2f8b8c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.704924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.705677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.705972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.706367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.707099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.709055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.756220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2052] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:00.760214 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.772303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b2218-5ba3-4bfd-9556-03834a8acf11 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.832308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.832980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.833262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.833538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.834220 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.836230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.855832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1033/2053] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:00 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:00.860216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.873016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f156e-0c2c-4606-ae7c-986354b0db99 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.878700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.879650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.880000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.880380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.881195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.882877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.896653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2054] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:00.909030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.921475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1746f-b3a1-4016-9555-68c653033a09 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.927537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36169c3-8dd9-4bc8-a315-646fd35502f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.928231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36169c3-8dd9-4bc8-a315-646fd35502f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.928519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36169c3-8dd9-4bc8-a315-646fd35502f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.928794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36169c3-8dd9-4bc8-a315-646fd35502f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:00.929877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36169c3-8dd9-4bc8-a315-646fd35502f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:00.931116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36169c3-8dd9-4bc8-a315-646fd35502f6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:00.948318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1034/2055] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:00.949979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:00.963364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bef2f-60bc-479f-81d9-854d2b9ed752 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:00.973058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:00.974120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:00.974477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:00.984423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] PATH_INFO: `/v3/projects/d1dc6d2fb622420f82505c82c7c98803/users/1784034b34384feaa501923ee1fc2d00/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.001202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dc6d2fb622420f82505c82c7c98803, user_id=1784034b34384feaa501923ee1fc2d00, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.003932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.019760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2056] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:00 2022] PUT /identity/v3/projects/d1dc6d2fb622420f82505c82c7c98803/users/1784034b34384feaa501923ee1fc2d00/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:01.031350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.032272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.032554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.065899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] MFA Rules not processed for user `1784034b34384feaa501923ee1fc2d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:01.067125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:01.083531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.085340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:01.146449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:01.154801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1035/2057] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:01.158869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.168688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aec4e-7cf5-40b3-b370-c47c7d290147 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.176739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.177502 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.177911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.178752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.182330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:01.237906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dfbf4-2691-40de-9797-98c8f2eae6cf None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.247696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.248976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.249688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.249989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.251630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.253959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.276681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.278366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.293314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1036/2058] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:01 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:01.294018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2059] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:01.297473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.382761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3bf8f-37c5-4563-ac77-a69498048572 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.389026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.390161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.392974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.393405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.394378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.396873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.431464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1037/2060] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:01 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:01.434755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.454301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49079e5-5d41-4b66-9196-66c7b651cd74 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.468656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.469891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.470180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.470495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.471404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.477035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.507273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2061] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:01.508993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.526413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24048b78-b466-4f93-a5c3-911e58982e33 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.543607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.544585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.544871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.545161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] PATH_INFO: `/v3/projects/d8de84d64c5241f2b6f879c72219721d/users/76bfe97bb9394df68e51cd50213c4b46/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.571497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8de84d64c5241f2b6f879c72219721d, user_id=76bfe97bb9394df68e51cd50213c4b46, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.574071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.592809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1038/2062] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:01 2022] PUT /identity/v3/projects/d8de84d64c5241f2b6f879c72219721d/users/76bfe97bb9394df68e51cd50213c4b46/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:01.597007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.622334 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617f014-1a79-40e5-9cc1-a9c0282d9d7f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.662603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:01.724230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac5638-3ed0-40a3-b6ef-09d528f634b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.725290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5638-3ed0-40a3-b6ef-09d528f634b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.726353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5638-3ed0-40a3-b6ef-09d528f634b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.726801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5638-3ed0-40a3-b6ef-09d528f634b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.727557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac5638-3ed0-40a3-b6ef-09d528f634b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.733534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac5638-3ed0-40a3-b6ef-09d528f634b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.746096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2063] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:01.752699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:01.767435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ae5a-ca79-4448-a98d-5febeaa8ff53 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:01.782759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:01.787809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.788100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.788420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] PATH_INFO: `/v3/projects/d8de84d64c5241f2b6f879c72219721d/users/76bfe97bb9394df68e51cd50213c4b46/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.794001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8de84d64c5241f2b6f879c72219721d, user_id=76bfe97bb9394df68e51cd50213c4b46, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:01.801039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:01.830933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1039/2064] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:01 2022] PUT /identity/v3/projects/d8de84d64c5241f2b6f879c72219721d/users/76bfe97bb9394df68e51cd50213c4b46/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:01.832954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:01.833441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:01.833838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:01.893141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] MFA Rules not processed for user `76bfe97bb9394df68e51cd50213c4b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:01.894567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:02.046847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2065] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:02.197275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:02.242814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:02.274299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff712-1a6d-40d0-bd69-e16883bb7fdc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:02.278462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:02.279190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:02.279544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:02.279961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:02.290822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:02.292302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:02.303222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1040/2066] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:02.445808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:02.447203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:02.450151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:02.451974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:02.467412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2883056-3aad-40e8-93f6-f568e362dda3 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:02.472162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:02.472948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:02.473340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:02.473804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:02.577414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355c266-44b6-4ff2-9c61-9682b303f0f8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:02.583239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:02.583854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:02.584146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:02.584467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:02.584748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] PATH_INFO: `/v3/users/ebca8b1c31f14accad7de26fe7464922` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:02.585406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:02.585951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebca8b1c31f14accad7de26fe7464922)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:02.588106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:02.627352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1041/2067] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:02.646950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:02.650577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebca8b1c31f14accad7de26fe7464922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:02.651368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebca8b1c31f14accad7de26fe7464922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:02.654436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:02.655611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731df40d-3464-4e94-abf3-48c8ea60adf7 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:02.660799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:02.661580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:02.661871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:02.662162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:02.663512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebca8b1c31f14accad7de26fe7464922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:02.665367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:02.672789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebca8b1c31f14accad7de26fe7464922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:02.686915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-975e490b-33f1-409d-b989-0f133a61fd73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebca8b1c31f14accad7de26fe7464922'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:02.699046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2068] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:02 2022] DELETE /identity/v3/users/ebca8b1c31f14accad7de26fe7464922 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:02.704315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:02.704315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:02.704315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:02.739536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:02.741632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:02.763940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:02.765799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:02.781919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1042/2069] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:02.834275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2070] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:02.845280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:02.847851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:02.915092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab14052-c265-4d6b-90c1-d123c481f776 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:02.937458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:02.969413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:02.970075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:02.970357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:02.970631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:02.980395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:02.982125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:02.993839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1043/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:03.375010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:03.376585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:03.389684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d926666-bb3a-47a4-9ad4-4b061de29af6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:03.395416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:03.397062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:03.397697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:03.398065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] PATH_INFO: `/v3/projects/d7a403993b2b41f0b1807a5ab2e1f19e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:03.399297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7a403993b2b41f0b1807a5ab2e1f19e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:03.401343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:03.459297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7a403993b2b41f0b1807a5ab2e1f19e', 'request_id': 'req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.465506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7a403993b2b41f0b1807a5ab2e1f19e', 'request_id': 'req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.472550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7a403993b2b41f0b1807a5ab2e1f19e', 'request_id': 'req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.484125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] The token cache is being invalidate because project d7a403993b2b41f0b1807a5ab2e1f19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:03.484468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.485082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.486608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2072] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:03 2022] DELETE /identity/v3/projects/d7a403993b2b41f0b1807a5ab2e1f19e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:03.490752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:03.501898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18782b6-a080-442f-9fe1-730cc9aababb service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:03.562400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:03.563904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:03.564361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:03.564644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] PATH_INFO: `/v3/users/907bdfaf1aaf492ebf8e761ab6750d02` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:03.565911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=907bdfaf1aaf492ebf8e761ab6750d02)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:03.568261 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:03.626864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907bdfaf1aaf492ebf8e761ab6750d02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.637081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907bdfaf1aaf492ebf8e761ab6750d02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.637730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907bdfaf1aaf492ebf8e761ab6750d02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.651734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907bdfaf1aaf492ebf8e761ab6750d02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.660605 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907bdfaf1aaf492ebf8e761ab6750d02'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:03.667377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1044/2073] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:03 2022] DELETE /identity/v3/users/907bdfaf1aaf492ebf8e761ab6750d02 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:03.979919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:03.981418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:03.995594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819d1f-e484-43fb-94b2-d6655eb4e6d8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:04.050383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:04.064332 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:04.065229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:04.065528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:04.066185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] PATH_INFO: `/v3/projects/e8e4bb9ac30b4559910fefc7f2e91a98` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:04.066999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e4bb9ac30b4559910fefc7f2e91a98)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:04.068929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:04.123142 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e4bb9ac30b4559910fefc7f2e91a98', 'request_id': 'req-a51f470a-ec48-4177-81f1-e55f5947202b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.131998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e4bb9ac30b4559910fefc7f2e91a98', 'request_id': 'req-a51f470a-ec48-4177-81f1-e55f5947202b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.142184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e4bb9ac30b4559910fefc7f2e91a98', 'request_id': 'req-a51f470a-ec48-4177-81f1-e55f5947202b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.153922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] The token cache is being invalidate because project e8e4bb9ac30b4559910fefc7f2e91a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:04.154215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.154860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.156469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2074] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:03 2022] DELETE /identity/v3/projects/e8e4bb9ac30b4559910fefc7f2e91a98 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:04.160264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:04.161940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:04.175045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc9440-0542-46f0-a1c7-44d5a9a77eb3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:04.234462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:04.237829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:04.242674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:04.279349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:04.280439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:04.280760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:04.281044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] PATH_INFO: `/v3/users/987220fe455f40bd98ae799296ede4da` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:04.282527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=987220fe455f40bd98ae799296ede4da)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:04.290475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:04.343531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f470a-ec48-4177-81f1-e55f5947202b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:04.353679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:04.355328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:04.356019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:04.356467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] PATH_INFO: `/v3/users/609f74b4a148429ba62059cb091c23d7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:04.358368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] RBAC: Authorizing `identity:delete_user(user_id=609f74b4a148429ba62059cb091c23d7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:04.370774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:04.382857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987220fe455f40bd98ae799296ede4da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.395547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987220fe455f40bd98ae799296ede4da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.396331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987220fe455f40bd98ae799296ede4da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.414497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987220fe455f40bd98ae799296ede4da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.429180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4844a0e9-6c87-43ef-92dc-84a339d9e826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987220fe455f40bd98ae799296ede4da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.444232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1045/2075] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:04 2022] DELETE /identity/v3/users/987220fe455f40bd98ae799296ede4da => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:04.446717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609f74b4a148429ba62059cb091c23d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.448971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609f74b4a148429ba62059cb091c23d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.459917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609f74b4a148429ba62059cb091c23d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.470942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609f74b4a148429ba62059cb091c23d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.503238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609f74b4a148429ba62059cb091c23d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:04.528723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:04.544901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2076] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:04 2022] DELETE /identity/v3/users/609f74b4a148429ba62059cb091c23d7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:04.545081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:04.545524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:04.545524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:04.589091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:04.592805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:04.733020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1046/2077] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:04.744487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:04.746130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:04.830103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5b014-3a07-4015-ba9e-2e0985b802be admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:04.834824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:04.836009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:04.836403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:04.836758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:04.853210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:04.859307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:04.885718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2078] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:04.996138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:04.998187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.018095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77f083-9c86-4d93-9818-07e354e83583 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.026463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.026664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.027238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.027238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] PATH_INFO: `/v3/users/9403835e4424431fa26c09abe25ec081` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.030139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9403835e4424431fa26c09abe25ec081)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.033310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.054063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.056153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.122932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdcd9a-92f9-4a13-9574-5ab1ed7dcc77 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.132886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.132886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.132886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.132886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] PATH_INFO: `/v3/projects/687ae1f508d44a758b62dab6ad8ee1ac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.134580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687ae1f508d44a758b62dab6ad8ee1ac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.136673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.140161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9403835e4424431fa26c09abe25ec081'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.151491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9403835e4424431fa26c09abe25ec081'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.153098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9403835e4424431fa26c09abe25ec081'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.162925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:05.184962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9403835e4424431fa26c09abe25ec081'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.199396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687ae1f508d44a758b62dab6ad8ee1ac', 'request_id': 'req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.204515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ec3cb9bb-d883-4e6d-ad05-39f02250a8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9403835e4424431fa26c09abe25ec081'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.206241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687ae1f508d44a758b62dab6ad8ee1ac', 'request_id': 'req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.218555 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687ae1f508d44a758b62dab6ad8ee1ac', 'request_id': 'req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.222301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1047/2079] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:04 2022] DELETE /identity/v3/users/9403835e4424431fa26c09abe25ec081 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:05.227641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.227969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.228645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.246736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] The token cache is being invalidate because project 687ae1f508d44a758b62dab6ad8ee1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:05.250927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.251676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:05.252361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-77ded32e-1ca0-4bb4-9f97-2ec3d4645903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.253468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.260015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2080] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:05 2022] DELETE /identity/v3/projects/687ae1f508d44a758b62dab6ad8ee1ac => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:05.270022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.270624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.270969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.293905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:05.293905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.388039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1048/2081] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:05.388912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.392426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.407205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b808621-c63b-47e8-b425-5c7e0bbb786f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.412251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.413100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.413388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.413670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] PATH_INFO: `/v3/projects/eb1df3e1bed34d18baa0dbf653bb2b8b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.414981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1df3e1bed34d18baa0dbf653bb2b8b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.417565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.426627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2082] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:05.427308 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.428642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.473786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:05.480841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1df3e1bed34d18baa0dbf653bb2b8b', 'request_id': 'req-2214edf3-7100-4d80-9d2b-84e4cc8bc203'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.492057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1df3e1bed34d18baa0dbf653bb2b8b', 'request_id': 'req-2214edf3-7100-4d80-9d2b-84e4cc8bc203'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.503797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1df3e1bed34d18baa0dbf653bb2b8b', 'request_id': 'req-2214edf3-7100-4d80-9d2b-84e4cc8bc203'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.512478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2485ad-d852-4bcc-9d8e-ee178a18b3fe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.515303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] The token cache is being invalidate because project eb1df3e1bed34d18baa0dbf653bb2b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:05.515599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.516431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-2214edf3-7100-4d80-9d2b-84e4cc8bc203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.518039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1049/2083] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:05 2022] DELETE /identity/v3/projects/eb1df3e1bed34d18baa0dbf653bb2b8b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:05.519442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.519559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.519820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.559985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:05.562234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.580338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.581130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.581437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.581927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.594766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.681886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1050/2084] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:05.686753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.689169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.699995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.701820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.703480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:05.717387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30f438-19ec-449d-801b-aa0b285d0a85 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.718066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2085] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:05.719822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.731325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3e71-1d70-453b-b53b-70fac693a491 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.734183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.734976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.735265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.735552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] PATH_INFO: `/v3/users/95ed66bab9fb4eebb5af01cfd098f4ba` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.736989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ed66bab9fb4eebb5af01cfd098f4ba)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.739113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.745714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.746678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.747661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.748387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.749586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:05.750619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.757130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.781151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2086] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:05.783385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.796869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbee29-e508-4680-9d8a-93a0a5a43b6e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:05.813381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:05.813381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:05.813381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:05.813381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:05.813836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:05.813836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:05.840083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ed66bab9fb4eebb5af01cfd098f4ba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.856156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ed66bab9fb4eebb5af01cfd098f4ba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.856905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2087] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:05 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:05.857570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ed66bab9fb4eebb5af01cfd098f4ba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.861330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:05.864354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:05.878053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ed66bab9fb4eebb5af01cfd098f4ba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.886479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ed66bab9fb4eebb5af01cfd098f4ba'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:05.895742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1051/2088] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:05 2022] DELETE /identity/v3/users/95ed66bab9fb4eebb5af01cfd098f4ba => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:05.954024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:05.966321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8d66d-7a93-49ad-b173-8d45c320bbe4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.102002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.103314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.103788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.104259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.105106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.107197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.136373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2089] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:05 2022] POST /identity/v3/users => generated 287 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:06.141264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.154302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07db14-8d03-4359-8e4e-0f62cb49077e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.181709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.182467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.182768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.184200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.185001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.186822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.190266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.192680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.212294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed09098-4443-4b44-b68e-3f4e3b83ce7e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.216407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1052/2090] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:06.218162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.220955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.222154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.222268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.222505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] PATH_INFO: `/v3/projects/508ea692359b410b956592ee02507b0d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.223990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=508ea692359b410b956592ee02507b0d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.226268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.240367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763ec5-36a6-4798-9ca8-34a084f94930 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.249738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.250610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.250959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.251363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.252249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.255245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.273698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1053/2091] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:06.273698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.273976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.293229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39248f45-84e7-46ea-ba94-61dafe961c82 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.308814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '508ea692359b410b956592ee02507b0d', 'request_id': 'req-e02be797-a965-431b-9971-ae80ec6a0cf2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.318005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '508ea692359b410b956592ee02507b0d', 'request_id': 'req-e02be797-a965-431b-9971-ae80ec6a0cf2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.329405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '508ea692359b410b956592ee02507b0d', 'request_id': 'req-e02be797-a965-431b-9971-ae80ec6a0cf2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.348978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] The token cache is being invalidate because project 508ea692359b410b956592ee02507b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:06.349439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.350404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.351869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2092] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:06 2022] DELETE /identity/v3/projects/508ea692359b410b956592ee02507b0d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:06.352811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.357084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.394064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.395146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.395723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.396064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] PATH_INFO: `/v3/users/a40cfdd7ddce4cd6b313ff889cf032d2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.399207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a40cfdd7ddce4cd6b313ff889cf032d2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.401706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.447914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02be797-a965-431b-9971-ae80ec6a0cf2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.453108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.454902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.455192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.455478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] PATH_INFO: `/v3/projects/0a3fe8942a3b42bdbc8e5a1c266faacc/users/4626b73e62fc44cda6e1c5da39e81752/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.456349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a40cfdd7ddce4cd6b313ff889cf032d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.468846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a40cfdd7ddce4cd6b313ff889cf032d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.469619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a40cfdd7ddce4cd6b313ff889cf032d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.471288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:06.472009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3fe8942a3b42bdbc8e5a1c266faacc, user_id=4626b73e62fc44cda6e1c5da39e81752, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.474554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.483833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:06.486190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a40cfdd7ddce4cd6b313ff889cf032d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.490320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2093] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:06 2022] PUT /identity/v3/projects/0a3fe8942a3b42bdbc8e5a1c266faacc/users/4626b73e62fc44cda6e1c5da39e81752/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:06.493798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ffb98098-d3ac-4f61-8219-2333af113102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a40cfdd7ddce4cd6b313ff889cf032d2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.493930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.496071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.500847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1054/2094] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:06 2022] DELETE /identity/v3/users/a40cfdd7ddce4cd6b313ff889cf032d2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:06.503718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cbe60-7b78-46a1-b71d-a5e86f35ac65 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.503718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cbe60-7b78-46a1-b71d-a5e86f35ac65 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.503718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cbe60-7b78-46a1-b71d-a5e86f35ac65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.524399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b59cbe60-7b78-46a1-b71d-a5e86f35ac65 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:06.525996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b59cbe60-7b78-46a1-b71d-a5e86f35ac65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.582531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc604b-1c41-4368-9b7f-e5984a38081d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.592413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1055/2095] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:06.593708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454a6d1-bff4-4c63-8133-d199b469e4df None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.594527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454a6d1-bff4-4c63-8133-d199b469e4df None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.594804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454a6d1-bff4-4c63-8133-d199b469e4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.626902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f454a6d1-bff4-4c63-8133-d199b469e4df None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:06.628374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f454a6d1-bff4-4c63-8133-d199b469e4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.636717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1056/2096] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:06.641416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c69d-7c39-4592-917b-aced2162e83c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.641714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c69d-7c39-4592-917b-aced2162e83c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.642001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c69d-7c39-4592-917b-aced2162e83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.656897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.657767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.658224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.658509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] PATH_INFO: `/v3/projects/442f5b696a4a4997889b795573e48b7c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.659541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1a37c69d-7c39-4592-917b-aced2162e83c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:06.659694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=442f5b696a4a4997889b795573e48b7c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.660765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1a37c69d-7c39-4592-917b-aced2162e83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.661664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.668470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1057/2097] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:06.669691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.669980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.670635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.689459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:06.690635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.698956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1058/2098] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:06.702992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.704369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.717236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442f5b696a4a4997889b795573e48b7c', 'request_id': 'req-120611e4-d49a-41f7-adb7-b5fd47c111d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.724758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442f5b696a4a4997889b795573e48b7c', 'request_id': 'req-120611e4-d49a-41f7-adb7-b5fd47c111d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.733668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442f5b696a4a4997889b795573e48b7c', 'request_id': 'req-120611e4-d49a-41f7-adb7-b5fd47c111d0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.745931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] The token cache is being invalidate because project 442f5b696a4a4997889b795573e48b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:06.746219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.746898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:06.748483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2099] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:06 2022] DELETE /identity/v3/projects/442f5b696a4a4997889b795573e48b7c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:06.752157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.752157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.769463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f9f5e-3159-4341-9879-48f89fee4570 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.796011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:06.815600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120611e4-d49a-41f7-adb7-b5fd47c111d0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.819436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.820102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.820424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.822789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.824888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.838215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.838875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.839286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.839566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.841613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.868035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:06.900968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.902552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.919958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.920377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2100] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:06.921111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.923084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.929834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1ebfe-d2fb-4a0b-a105-afb24212d59a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.935759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f387-d1df-4218-98ee-680b292ab52a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.945711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.946388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.946675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.946950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.947683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:06.947993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.949650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1059/2101] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:06.949933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.951653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ce583-439d-45d1-aba1-6f8965be68c9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.952156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ce583-439d-45d1-aba1-6f8965be68c9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.952669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ce583-439d-45d1-aba1-6f8965be68c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.965489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2102] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:06.970006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:06.973373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c82ce583-439d-45d1-aba1-6f8965be68c9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:06.974570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c82ce583-439d-45d1-aba1-6f8965be68c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:06.983370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6d9da-1445-47c6-af31-6908c0259064 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:06.988255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1060/2103] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:06.992066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:06.992864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.993415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:06.993908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:06.994771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:06.995619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:06.996744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:06.999899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.000556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.032887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:07.034333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.040366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2104] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:07.044951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.046242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1061/2105] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:07.050389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.059931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bca98-43f7-4fa1-8a6d-39ce4d81d017 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.132080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.133203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.133587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.134113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.134952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:07.135441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.137546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.147274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261bc13-1d8f-4b6b-9cc0-2236710e2ae3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.152705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.152898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1062/2106] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:32:07.153300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.153754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.154115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.154884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.157987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.158701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.171256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b5c8-5175-4edf-9707-a7797c4dbaad admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.182110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.183979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.184368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.184826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.185859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.189752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2107] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:07 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:07.190118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.191106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.204464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baeb28-d012-4614-88f5-583a27c11c1d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.210401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.211066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.211357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.211635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.212507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.214271 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.226195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1063/2108] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:07 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:07.226195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.236926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2109] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:07.237925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.240676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.252532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196c4a7-fa55-4587-84fd-4a88e156d63a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.289270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:07.344070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.344855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.345193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.345478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.346339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.349784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.350544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9728818-faf5-461f-9e0c-36cbe2e46a8f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.356461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.358302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.359489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.360373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] PATH_INFO: `/v3/users/f3f829e06fa44c4d88bd65474444c5de` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.362339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f829e06fa44c4d88bd65474444c5de)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.364382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.377860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2110] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:07 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:07.379028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.401246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d3bf-96e5-49f5-a863-f9db5013f2ea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.408200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:07.417658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.417658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.417658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.419632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.422871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.425928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.450326 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2111] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:07.451290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.473970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d787bf8-2942-46f7-93da-c32f5d3b20df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.481246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.482175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.482292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.482649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.483301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.483806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f829e06fa44c4d88bd65474444c5de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.485616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.494622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f829e06fa44c4d88bd65474444c5de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.495243 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f829e06fa44c4d88bd65474444c5de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.508151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2112] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:07.513020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.515235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.519803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f829e06fa44c4d88bd65474444c5de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.530481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906723c-1e3a-4369-93f7-1f337fcc36ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.533895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f829e06fa44c4d88bd65474444c5de'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.540090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.541035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.541290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.541540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] PATH_INFO: `/v3/projects/f41c49a69e3f474598c1ebae85293551` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.543611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41c49a69e3f474598c1ebae85293551)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.545808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.557513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1064/2113] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:07 2022] DELETE /identity/v3/users/f3f829e06fa44c4d88bd65474444c5de => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:07.557513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.557513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.602558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41c49a69e3f474598c1ebae85293551', 'request_id': 'req-e76ea62a-daa8-4006-beff-5d21c2025cba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.622854 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41c49a69e3f474598c1ebae85293551', 'request_id': 'req-e76ea62a-daa8-4006-beff-5d21c2025cba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.634262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41c49a69e3f474598c1ebae85293551', 'request_id': 'req-e76ea62a-daa8-4006-beff-5d21c2025cba'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.659688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] The token cache is being invalidate because project f41c49a69e3f474598c1ebae85293551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:07.660775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.662660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:07.666275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2114] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:07 2022] DELETE /identity/v3/projects/f41c49a69e3f474598c1ebae85293551 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:07.667548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.671109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.673461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f486a-7292-4354-a785-59190f0622fb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.758512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3879177-1a78-4af6-8707-e0c1fbd0befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.759504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879177-1a78-4af6-8707-e0c1fbd0befc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.759900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879177-1a78-4af6-8707-e0c1fbd0befc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.760278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879177-1a78-4af6-8707-e0c1fbd0befc admin admin] PATH_INFO: `/v3/projects/14d1003cc7f84a0f97f1b0b0bdb52b58/users/86a53b568bc54aa2a93fa85f3979436f/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.783230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3879177-1a78-4af6-8707-e0c1fbd0befc admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d1003cc7f84a0f97f1b0b0bdb52b58, user_id=86a53b568bc54aa2a93fa85f3979436f, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.787520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3879177-1a78-4af6-8707-e0c1fbd0befc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.799566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ea62a-daa8-4006-beff-5d21c2025cba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.812371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1065/2115] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:07 2022] PUT /identity/v3/projects/14d1003cc7f84a0f97f1b0b0bdb52b58/users/86a53b568bc54aa2a93fa85f3979436f/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:07.815112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217cf2-a878-4f13-ad2a-82c16032edc0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.815465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217cf2-a878-4f13-ad2a-82c16032edc0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.815806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df217cf2-a878-4f13-ad2a-82c16032edc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.838202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-df217cf2-a878-4f13-ad2a-82c16032edc0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:07.839513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-df217cf2-a878-4f13-ad2a-82c16032edc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.856097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:07.861673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:07.896474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.897452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.897901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.898294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.899253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.900974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.913036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2116] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:07.914122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:07.916357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:07.922775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1066/2117] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:07.923822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.924273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.924598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.932846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc813f-52a4-47b8-b104-4fff008b44b2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:07.938594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:07.939563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:07.940024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:07.940527 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] PATH_INFO: `/v3/users/1784034b34384feaa501923ee1fc2d00` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:07.941953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1784034b34384feaa501923ee1fc2d00)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:07.944615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:07.962738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] MFA Rules not processed for user `86a53b568bc54aa2a93fa85f3979436f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:07.964732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.015357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1784034b34384feaa501923ee1fc2d00'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.018118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1784034b34384feaa501923ee1fc2d00'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.027345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1784034b34384feaa501923ee1fc2d00'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.036655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1784034b34384feaa501923ee1fc2d00'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.049143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1067/2118] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:08.050161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.053324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1784034b34384feaa501923ee1fc2d00'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.060610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.065886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2119] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:07 2022] DELETE /identity/v3/users/1784034b34384feaa501923ee1fc2d00 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:08.065886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.065886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.095508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d22b8c-bea1-4859-8947-288ddec00fb3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.184494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602d01c-2491-4f68-b596-130eee36d3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:08.185679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602d01c-2491-4f68-b596-130eee36d3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.186146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602d01c-2491-4f68-b596-130eee36d3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.186563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602d01c-2491-4f68-b596-130eee36d3b9 admin admin] PATH_INFO: `/v3/projects/0f705d4065054c75b90eaaaf6d77e275/users/87de392663e541fd87dd6a94288d3477/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.202901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2259d22-8501-4325-8df3-4a842edbf476 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.216296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:08.216846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.217361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.217668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602d01c-2491-4f68-b596-130eee36d3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f705d4065054c75b90eaaaf6d77e275, user_id=87de392663e541fd87dd6a94288d3477, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:08.217822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.220152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.220860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602d01c-2491-4f68-b596-130eee36d3b9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:08.237837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:08.239498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:08.240716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1068/2120] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:08 2022] PUT /identity/v3/projects/0f705d4065054c75b90eaaaf6d77e275/users/87de392663e541fd87dd6a94288d3477/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:08.242475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d86c6-284e-46e6-82d8-017e6c13aaeb None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.242828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d86c6-284e-46e6-82d8-017e6c13aaeb None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.243185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d86c6-284e-46e6-82d8-017e6c13aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.262740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2121] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:08.263700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.265138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.280017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-633d86c6-284e-46e6-82d8-017e6c13aaeb None None] MFA Rules not processed for user `4626b73e62fc44cda6e1c5da39e81752`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:08.281432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-633d86c6-284e-46e6-82d8-017e6c13aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.323211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:08.329266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:08.353697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238c4e-388a-4569-8956-33279b1bf306 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.375568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1069/2122] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:08.377342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.377646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.378121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.424485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:08.425001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.425350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.425640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.425899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] MFA Rules not processed for user `87de392663e541fd87dd6a94288d3477`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:08.427296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.521431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:08.524631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1070/2123] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:08.525648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.527089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.528787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:08.544938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2124] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:08.545937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.547514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:08.635598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1cf7e-e69f-47de-a178-9780f2ada848 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.640518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:08.642098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.642387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.642660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.643445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:08.645346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:08.679686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1071/2125] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:08 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:08.681483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.689390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852df3ff-8084-472d-b66d-35148c6445c8 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.756994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:08.794681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:08.797560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc837f1-6488-4f26-8e03-702474870d94 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.802740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:08.803599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.803938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.804250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.804951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:08.806564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:08.806980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:08.807512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:08.807801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:08.808156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] PATH_INFO: `/v3/projects/d1dc6d2fb622420f82505c82c7c98803` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:08.809679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1dc6d2fb622420f82505c82c7c98803)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:08.811579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:08.842028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1072/2126] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:08 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:08.842028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.881582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1dc6d2fb622420f82505c82c7c98803', 'request_id': 'req-c34a89ae-3c21-4ba0-8365-770b016f6f4c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.888330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1dc6d2fb622420f82505c82c7c98803', 'request_id': 'req-c34a89ae-3c21-4ba0-8365-770b016f6f4c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.909418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1dc6d2fb622420f82505c82c7c98803', 'request_id': 'req-c34a89ae-3c21-4ba0-8365-770b016f6f4c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.922412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d9ce6-0dbb-49d6-8eda-b732f3af0e63 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:08.932634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] The token cache is being invalidate because project d1dc6d2fb622420f82505c82c7c98803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:08.933099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.934590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:08.945799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2127] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:08 2022] DELETE /identity/v3/projects/d1dc6d2fb622420f82505c82c7c98803 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:08.947075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:08.949932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.010397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.011191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.011648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.012120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.012951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.015282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.040716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1073/2128] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:08 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:09.040924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a89ae-3c21-4ba0-8365-770b016f6f4c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.041010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.042373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.043095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.044126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.044580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.045003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] PATH_INFO: `/v3/projects/afc83e5f26bc4db7a6c7fc4b223d1518` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.046414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc83e5f26bc4db7a6c7fc4b223d1518)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.049306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.057002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76868ea7-df37-4bc6-8ea0-2da1b97483f0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.061673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.062339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.063204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.063484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.064170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.066157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.086494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1074/2129] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:09 2022] POST /identity/v3/users => generated 295 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:09.087309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.088875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.100195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc83e5f26bc4db7a6c7fc4b223d1518', 'request_id': 'req-14648ff1-6f98-4226-9426-b5374b239366'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.100603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c06663-99ba-41b6-8b3f-e9dc4b0210cd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.106255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc83e5f26bc4db7a6c7fc4b223d1518', 'request_id': 'req-14648ff1-6f98-4226-9426-b5374b239366'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.114410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc83e5f26bc4db7a6c7fc4b223d1518', 'request_id': 'req-14648ff1-6f98-4226-9426-b5374b239366'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.127208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] The token cache is being invalidate because project afc83e5f26bc4db7a6c7fc4b223d1518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:09.127524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.128249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.130055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2130] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:08 2022] DELETE /identity/v3/projects/afc83e5f26bc4db7a6c7fc4b223d1518 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.130840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.132263 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.152386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:09.152690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:09.167684 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.168607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.168896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.169176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] PATH_INFO: `/v3/users/76bfe97bb9394df68e51cd50213c4b46` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.170374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76bfe97bb9394df68e51cd50213c4b46)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.172983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.202171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648ff1-6f98-4226-9426-b5374b239366 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.213537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.214373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.214882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.215150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.215906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.217708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.230896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2131] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:09.232167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.234215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.237571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76bfe97bb9394df68e51cd50213c4b46'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.246269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76bfe97bb9394df68e51cd50213c4b46'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.247213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76bfe97bb9394df68e51cd50213c4b46'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.252723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01e3da-d1ec-4ac1-8080-52697f3c32b8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.257676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.258350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.258643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.258931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.259675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.261407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.264023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76bfe97bb9394df68e51cd50213c4b46'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.271958 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2132] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:09.272105 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76bfe97bb9394df68e51cd50213c4b46'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.272849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.276064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.283355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1075/2133] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:09 2022] DELETE /identity/v3/users/76bfe97bb9394df68e51cd50213c4b46 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.283355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.283355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.301759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d95e5-2059-4ec3-9281-6a7365828656 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.382368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308ecfc-69d5-425c-b8f0-05c1722c05c7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.393268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.394483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.394868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.395254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] PATH_INFO: `/v3/projects/3bde225a82cd49519ed903a76147ddfe/users/1e2c9c4fda4447d898a2d0aafb6c9959/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.424416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bde225a82cd49519ed903a76147ddfe, user_id=1e2c9c4fda4447d898a2d0aafb6c9959, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.428721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.450423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.455650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1076/2134] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:09 2022] PUT /identity/v3/projects/3bde225a82cd49519ed903a76147ddfe/users/1e2c9c4fda4447d898a2d0aafb6c9959/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.456550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.456932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.457295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] PATH_INFO: `/v3/users/eb202ec3c6ad43b5aabeba01810b236f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.459071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb202ec3c6ad43b5aabeba01810b236f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.459773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.461299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.461599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.543943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb202ec3c6ad43b5aabeba01810b236f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.545421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb202ec3c6ad43b5aabeba01810b236f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.557102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb202ec3c6ad43b5aabeba01810b236f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.558676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:09.569794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb202ec3c6ad43b5aabeba01810b236f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.576577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129b8b6-a3df-4258-b66c-048ff9a53327 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.582190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.583535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.583535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.583992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] PATH_INFO: `/v3/projects/05016b1c2cfb4e00a61ba5ab8d9b756d/users/78edec2b73164de48d4ccd3926651010/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.588568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb202ec3c6ad43b5aabeba01810b236f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.596764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2135] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:09 2022] DELETE /identity/v3/users/eb202ec3c6ad43b5aabeba01810b236f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.597664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.598945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.608576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05016b1c2cfb4e00a61ba5ab8d9b756d, user_id=78edec2b73164de48d4ccd3926651010, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.612599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.634903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1077/2136] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:09 2022] PUT /identity/v3/projects/05016b1c2cfb4e00a61ba5ab8d9b756d/users/78edec2b73164de48d4ccd3926651010/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.639374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.642894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.675606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:09.753366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a236f2f-a426-49f6-ab93-762e264ce0d9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.761309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b704866-3db3-408a-9f73-82debd83bcc3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.764017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.764842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.765204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.766104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.767045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.769580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.779537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.784717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.785167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.785590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.786536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.788646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.795081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2137] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:09.795081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.795841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:09.802069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1078/2138] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:09.802988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.815119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee0f47-ec9d-464a-84d2-49b696dd4976 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.815581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5d16-ff9b-4205-8776-2da5f2d34abb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.821503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.822673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.822792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:09.823048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.823431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] PATH_INFO: `/v3/projects/3bde225a82cd49519ed903a76147ddfe/users/1e2c9c4fda4447d898a2d0aafb6c9959/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.823722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.824013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.824858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] PATH_INFO: `/v3/projects/d8de84d64c5241f2b6f879c72219721d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:09.825755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bde225a82cd49519ed903a76147ddfe, user_id=1e2c9c4fda4447d898a2d0aafb6c9959, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.826697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8de84d64c5241f2b6f879c72219721d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:09.828787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.828902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:09.852764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1079/2139] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:09 2022] PUT /identity/v3/projects/3bde225a82cd49519ed903a76147ddfe/users/1e2c9c4fda4447d898a2d0aafb6c9959/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.857675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:09.912672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8de84d64c5241f2b6f879c72219721d', 'request_id': 'req-2b6cbb7a-1478-432c-895d-18f7f02be91e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.920262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8de84d64c5241f2b6f879c72219721d', 'request_id': 'req-2b6cbb7a-1478-432c-895d-18f7f02be91e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.931800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8de84d64c5241f2b6f879c72219721d', 'request_id': 'req-2b6cbb7a-1478-432c-895d-18f7f02be91e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.949951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] The token cache is being invalidate because project d8de84d64c5241f2b6f879c72219721d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:09.950410 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.951387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b6cbb7a-1478-432c-895d-18f7f02be91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:09.954545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971be78-c24b-49bf-802e-a70e069a68a7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:09.958686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2140] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:09 2022] DELETE /identity/v3/projects/d8de84d64c5241f2b6f879c72219721d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:09.966049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feab0c-ca14-4210-b4d0-1a88325d12a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:09.967071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feab0c-ca14-4210-b4d0-1a88325d12a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:09.967367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4feab0c-ca14-4210-b4d0-1a88325d12a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.003961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4feab0c-ca14-4210-b4d0-1a88325d12a5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:10.005913 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4feab0c-ca14-4210-b4d0-1a88325d12a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.085914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:10.087839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.088516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.088629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] PATH_INFO: `/v3/projects/05016b1c2cfb4e00a61ba5ab8d9b756d/users/78edec2b73164de48d4ccd3926651010/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.093470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=05016b1c2cfb4e00a61ba5ab8d9b756d, user_id=78edec2b73164de48d4ccd3926651010, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:10.097727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:10.107532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:10.136771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1080/2141] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:09 2022] PUT /identity/v3/projects/05016b1c2cfb4e00a61ba5ab8d9b756d/users/78edec2b73164de48d4ccd3926651010/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:10.137617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2142] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:10.141083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:10.142216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.148897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca917f-ed83-4522-8277-d1f50bda114b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.151344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca917f-ed83-4522-8277-d1f50bda114b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.151651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca917f-ed83-4522-8277-d1f50bda114b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.191164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-53ca917f-ed83-4522-8277-d1f50bda114b None None] MFA Rules not processed for user `78edec2b73164de48d4ccd3926651010`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:10.192646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-53ca917f-ed83-4522-8277-d1f50bda114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.225262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92790c-35a2-4acb-875e-3463e1843bee admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:10.231880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:10.232718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.233007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.233415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] PATH_INFO: `/v3/projects/f20f12cbb1854376b67d5b39fb2c86c3` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.235193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20f12cbb1854376b67d5b39fb2c86c3)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:10.236966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:10.238896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:10.333456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20f12cbb1854376b67d5b39fb2c86c3', 'request_id': 'req-848d599d-ee35-45cd-b567-63bad0153cde'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.348592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20f12cbb1854376b67d5b39fb2c86c3', 'request_id': 'req-848d599d-ee35-45cd-b567-63bad0153cde'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.350818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2143] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:10.353288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.353881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.354144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.362709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20f12cbb1854376b67d5b39fb2c86c3', 'request_id': 'req-848d599d-ee35-45cd-b567-63bad0153cde'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.393269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:10.394651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] The token cache is being invalidate because project f20f12cbb1854376b67d5b39fb2c86c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:10.395113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.395845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.397168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.400306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1081/2144] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:10 2022] DELETE /identity/v3/projects/f20f12cbb1854376b67d5b39fb2c86c3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:10.400306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:10.401551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.529401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d599d-ee35-45cd-b567-63bad0153cde admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:10.538946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:10.540357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.542702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.543000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.548578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.556416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2145] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:10.557296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:10.560664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.585974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab43fc7-0630-4a34-8485-018950c31cfe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:10.598899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:10.600875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.601467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.604007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] PATH_INFO: `/v3/users/266eed857b774741a00a16e3a470b2bd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.607641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266eed857b774741a00a16e3a470b2bd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:10.613512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:10.631675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:10.658442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:10.660217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:10.693336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1082/2146] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:10.694579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:10.727673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b210-a268-4a4a-8055-9389e60af6ce service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:10.743711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266eed857b774741a00a16e3a470b2bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.744801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266eed857b774741a00a16e3a470b2bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.746160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:10.746997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.747364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.747720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.748793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:10.749494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:10.755650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:10.764195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266eed857b774741a00a16e3a470b2bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.777988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266eed857b774741a00a16e3a470b2bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.788005 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1083/2147] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:10.792534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:10.797177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.808285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52ec2fce-48d9-4e0c-a0f5-7ba2bdb7a913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266eed857b774741a00a16e3a470b2bd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:10.823643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2148] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:10 2022] DELETE /identity/v3/users/266eed857b774741a00a16e3a470b2bd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:10.829257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edceb7a-2492-4aef-8055-573a04121b42 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:10.892160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.892456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.892739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.964942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] MFA Rules not processed for user `1e2c9c4fda4447d898a2d0aafb6c9959`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:10.966919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:10.973103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:10.973781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:10.974067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:10.977365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:10.978138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:10.980093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.033193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:11.044836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1084/2149] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:10 2022] POST /identity/v3/projects => generated 377 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:11.049833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.120812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2150] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:11.188655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.193330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.210539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd062f8-b7ea-4292-9239-16ef7fee85ab admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.230951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.231664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.231958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.232301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.238808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.243751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.279714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1085/2151] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:11 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:11.283921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.296660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb5f83-5035-4bb4-93e7-5cb764a9e4d7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.303407 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.304239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.305193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.305485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.306374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.308480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.324451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2973d19-d7ff-40be-8469-02529c35f15e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.325746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1086/2152] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:11.326371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.337431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f28233-293c-45bc-823b-0dcf74d5a03a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.343428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.344476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.344754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.345043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.345783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.347873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.373934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1087/2153] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:11.378740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.398279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.399403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.399871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.400366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.402673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.410321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9a0f3-4de2-4725-b58f-04b7968f08f9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.425052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.426136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.426428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.426712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] PATH_INFO: `/v3/projects/5eba094c78f14adfbade06ea4f93065e/users/dd95c260fea24556b15707fde82b6833/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.450601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eba094c78f14adfbade06ea4f93065e, user_id=dd95c260fea24556b15707fde82b6833, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.453955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.468768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:11.478414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1088/2154] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:11 2022] PUT /identity/v3/projects/5eba094c78f14adfbade06ea4f93065e/users/dd95c260fea24556b15707fde82b6833/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:11.479404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.481836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.512013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.513570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ba043-964b-4d1e-b9a2-cd1968926aa0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.532411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2155] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:11.536064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.536531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.537159 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.568728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] MFA Rules not processed for user `dd95c260fea24556b15707fde82b6833`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:11.570181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.570685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d31802-f0d4-4fbf-a74e-011efd3ab0ac admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.576412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.579019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.579851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.580348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] PATH_INFO: `/v3/projects/29a0be97e73a46ddbaf4f2321b010bc0` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.581668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a0be97e73a46ddbaf4f2321b010bc0)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.583699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.600916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:11.643156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a0be97e73a46ddbaf4f2321b010bc0', 'request_id': 'req-08337a38-796b-4ac3-9c95-09c97daa6717'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:11.651907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a0be97e73a46ddbaf4f2321b010bc0', 'request_id': 'req-08337a38-796b-4ac3-9c95-09c97daa6717'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:11.654289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2156] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:11.661600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a0be97e73a46ddbaf4f2321b010bc0', 'request_id': 'req-08337a38-796b-4ac3-9c95-09c97daa6717'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:11.666146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.667405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.674239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] The token cache is being invalidate because project 29a0be97e73a46ddbaf4f2321b010bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:11.674710 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:11.676419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-08337a38-796b-4ac3-9c95-09c97daa6717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:11.681372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1089/2157] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:11 2022] DELETE /identity/v3/projects/29a0be97e73a46ddbaf4f2321b010bc0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:11.691465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac01dc-e904-49df-862b-5e127a0ad264 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.691465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac01dc-e904-49df-862b-5e127a0ad264 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.691465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac01dc-e904-49df-862b-5e127a0ad264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.712786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-03ac01dc-e904-49df-862b-5e127a0ad264 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:11.713935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-03ac01dc-e904-49df-862b-5e127a0ad264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.742549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079829b-1ef3-4a23-bf18-26371495588e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.813044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1090/2158] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:11.814306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.814306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.814306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.814306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.816855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.817158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.817456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.820113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.841182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:11.852411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:11.873839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1091/2159] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:11.878307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.899080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76249f-b123-4b8e-90b1-6a49de030c27 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.904112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.904811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.905095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.905371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.906130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:11.906436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.908195 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.924562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.925522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.928222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1092/2160] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:32:11.941379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:11.945133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fef94b-db57-41ae-b5ff-b1aa3e243531 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:11.948917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2161] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:11.950265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:11.950959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:11.951237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:11.951579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:11.952296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:11.954296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:11.959457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:12.010495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1093/2162] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:11 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:12.022359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:12.034914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf02964-fe71-4c65-bb95-2a88f82b0c91 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:12.069740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:12.096785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:12.097472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.097763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.098056 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.098756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:12.101104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:12.128224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2163] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:12 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:12.132211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:12.143600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6a299-fa83-4c02-9d2f-d517a25c8deb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:12.149134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:12.150149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.150559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.150927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.152008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:12.153760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:12.169429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1094/2164] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:12.173431 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:12.186092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db584c-0d29-49bc-a2ef-4aba491a9909 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:12.191110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b9644-4fd4-420d-80fe-9a9f1bc689bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:12.191829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9644-4fd4-420d-80fe-9a9f1bc689bd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.192122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9644-4fd4-420d-80fe-9a9f1bc689bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.192633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9644-4fd4-420d-80fe-9a9f1bc689bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.193494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b9644-4fd4-420d-80fe-9a9f1bc689bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:12.195080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b9644-4fd4-420d-80fe-9a9f1bc689bd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:12.208845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2165] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:12.212860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:12.223532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a15e01-6e2f-47cd-b244-1d34638eee8b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:12.228425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:12.229307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.229601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.230142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] PATH_INFO: `/v3/projects/22eee074e0a44037a063390ff9f47f64/users/8703a547d0694067af12620201822f50/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.257842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22eee074e0a44037a063390ff9f47f64, user_id=8703a547d0694067af12620201822f50, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:12.260826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:12.275455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1095/2166] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:12 2022] PUT /identity/v3/projects/22eee074e0a44037a063390ff9f47f64/users/8703a547d0694067af12620201822f50/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:12.298816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.299457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.299897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.336359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] MFA Rules not processed for user `8703a547d0694067af12620201822f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:12.337845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:12.430436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2167] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:12.473778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:12.475437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:12.588638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db7f23-356e-4ef8-b3e2-e7d80bb985d1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:12.592466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8c925-8c51-4788-a64d-babee694acf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:12.593125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8c925-8c51-4788-a64d-babee694acf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.593400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8c925-8c51-4788-a64d-babee694acf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.594197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8c925-8c51-4788-a64d-babee694acf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.603481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8c925-8c51-4788-a64d-babee694acf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:12.604986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8c925-8c51-4788-a64d-babee694acf2 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:12.614958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1096/2168] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:12.744809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:12.746804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:12.838885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6d010-1e52-4bc1-9aa5-72ca788caabd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:12.843528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:12.844991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.845277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.845677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] PATH_INFO: `/v3/users/86a53b568bc54aa2a93fa85f3979436f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:12.846877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a53b568bc54aa2a93fa85f3979436f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:12.849132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:12.912304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:12.914102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a53b568bc54aa2a93fa85f3979436f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:12.914939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a53b568bc54aa2a93fa85f3979436f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:12.935407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a53b568bc54aa2a93fa85f3979436f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:12.955052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a53b568bc54aa2a93fa85f3979436f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:12.988619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a53b568bc54aa2a93fa85f3979436f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:12.989043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2169] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:12 2022] DELETE /identity/v3/users/86a53b568bc54aa2a93fa85f3979436f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:12.994723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:12.995313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:12.995440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.020340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:13.021589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.052501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.054745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.100911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1097/2170] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:13.111125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.112525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.117941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7881-f1c1-47a2-b569-d265473472bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.121952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.122680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.124036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.124036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.125709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.177636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef2719-2b84-41d1-9e7a-47af210e5b3e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.184001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61c070-b748-4527-8ae2-0258d046c10e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.184965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61c070-b748-4527-8ae2-0258d046c10e service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.185256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61c070-b748-4527-8ae2-0258d046c10e service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.185542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61c070-b748-4527-8ae2-0258d046c10e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.195662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.197707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61c070-b748-4527-8ae2-0258d046c10e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.198751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.199272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61c070-b748-4527-8ae2-0258d046c10e service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.203619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:13.211601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2171] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:13.212090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1098/2172] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:13.500520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.502049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.516702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d0e88-d3c2-4f11-836b-9e65f7d3f634 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.522335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.523203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.523485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.523767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] PATH_INFO: `/v3/projects/14d1003cc7f84a0f97f1b0b0bdb52b58` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.524981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d1003cc7f84a0f97f1b0b0bdb52b58)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.526881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.576669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d1003cc7f84a0f97f1b0b0bdb52b58', 'request_id': 'req-70d9c2de-f503-4702-83de-41f25cd504b0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:13.582649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d1003cc7f84a0f97f1b0b0bdb52b58', 'request_id': 'req-70d9c2de-f503-4702-83de-41f25cd504b0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:13.589477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d1003cc7f84a0f97f1b0b0bdb52b58', 'request_id': 'req-70d9c2de-f503-4702-83de-41f25cd504b0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:13.591477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:13.604119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] The token cache is being invalidate because project 14d1003cc7f84a0f97f1b0b0bdb52b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:13.604655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:13.612622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-70d9c2de-f503-4702-83de-41f25cd504b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:13.614475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2173] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:13 2022] DELETE /identity/v3/projects/14d1003cc7f84a0f97f1b0b0bdb52b58 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:13.619328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.619717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.620116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.638260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:13.639454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.705552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1099/2174] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:13.714110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.714110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.714330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.733524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:13.734768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:13.742914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2175] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:13.749696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.759492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefeef3-e2ef-418d-afc9-cbaf30df75df None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.764066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.764750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.765038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.765328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.766588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:13.766894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.768527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.784502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1100/2176] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:13.785661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.798617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd9c9-2189-4227-a138-995c711e97ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.803239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.803919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.804230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.804536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.805223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.807113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.835151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2177] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:13 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:13.836440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.846931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677d5b6-59fb-482f-bb4b-44a692e7266c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.903101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.903757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.904040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.904353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.905044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.907039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.925883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1101/2178] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:13 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:13.930306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.942005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c56b44-0016-4f24-9504-2ced67552890 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.946779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.947490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.947783 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.948097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.948964 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.950474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.961610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2179] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:13.965588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:13.975426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6afbc-7ad8-4e90-b347-c0ac1ceddaec admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:13.980034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3cd6e-575b-4ff5-aed6-1232ed0b5847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:13.980713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3cd6e-575b-4ff5-aed6-1232ed0b5847 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:13.980998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3cd6e-575b-4ff5-aed6-1232ed0b5847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:13.981277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3cd6e-575b-4ff5-aed6-1232ed0b5847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:13.982003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3cd6e-575b-4ff5-aed6-1232ed0b5847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:13.983622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3cd6e-575b-4ff5-aed6-1232ed0b5847 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:13.994147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1102/2180] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:13.998103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:14.009748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cbb379-a0a7-4e9e-bfb4-c971afbb5835 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:14.014538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:14.015464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:14.016538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:14.017237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] PATH_INFO: `/v3/projects/7c5a1bf2e2af4328ab3f1b68b4160b7b/users/f832895e13784dfaa69b44505da8855b/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:14.035029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5a1bf2e2af4328ab3f1b68b4160b7b, user_id=f832895e13784dfaa69b44505da8855b, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:14.037765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:14.050873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2181] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:13 2022] PUT /identity/v3/projects/7c5a1bf2e2af4328ab3f1b68b4160b7b/users/f832895e13784dfaa69b44505da8855b/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:14.055757 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6fb9-290e-4cfb-b4b5-74d48e4ff35f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:14.056227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6fb9-290e-4cfb-b4b5-74d48e4ff35f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:14.056357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c6fb9-290e-4cfb-b4b5-74d48e4ff35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:14.074271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-840c6fb9-290e-4cfb-b4b5-74d48e4ff35f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:14.075441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-840c6fb9-290e-4cfb-b4b5-74d48e4ff35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:14.133257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1103/2182] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:14.143363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:14.145052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:14.200663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7521b7-36e6-4ee2-a309-a3e3aa7906dc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:14.204937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:14.205572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:14.205858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:14.206138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:14.215052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:14.216503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:14.225886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2183] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:14.870710 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2a6c3-5583-4f7d-a516-2298f84f28c5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:14.871067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2a6c3-5583-4f7d-a516-2298f84f28c5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:14.871414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2a6c3-5583-4f7d-a516-2298f84f28c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:14.878989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:14.902766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-98c2a6c3-5583-4f7d-a516-2298f84f28c5 None None] MFA Rules not processed for user `f832895e13784dfaa69b44505da8855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:14.904787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-98c2a6c3-5583-4f7d-a516-2298f84f28c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:14.976217 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1104/2184] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:14.987596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:14.989147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:15.049399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75473d3b-8f3e-4211-b3cc-d2d6b9820a5e service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:15.053105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:15.053752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:15.054044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:15.054335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:15.062952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:15.064416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:15.073262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2185] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:16.411259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:16.411803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:16.411803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:16.430290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] MFA Rules not processed for user `f832895e13784dfaa69b44505da8855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:16.431487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:16.438745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1105/2186] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:16.908812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:16.920570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:16.939874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9724308-cc40-4f57-9c66-252464ced8f3 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:16.949471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:16.951338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:16.951954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:16.952527 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:16.955153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:17.001176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:17.035397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:17.036878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe6c50-5192-4352-b32c-3ed2ee8cdf2f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:17.048338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2187] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:18.564035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:18.565553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:18.581310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04687693-f5a6-4c0a-9d28-c5011b46ba72 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:18.585754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:18.586624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:18.586910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:18.587187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] PATH_INFO: `/v3/users/8703a547d0694067af12620201822f50` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:18.588400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8703a547d0694067af12620201822f50)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:18.590362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:18.638563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8703a547d0694067af12620201822f50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:18.645498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8703a547d0694067af12620201822f50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:18.646114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8703a547d0694067af12620201822f50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:18.659122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8703a547d0694067af12620201822f50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:18.665897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8703a547d0694067af12620201822f50'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:18.671695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1106/2188] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:18 2022] DELETE /identity/v3/users/8703a547d0694067af12620201822f50 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:18.676127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:18.676617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:18.676711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:18.693638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:18.694828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:18.752987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2189] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:18.762405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:18.763811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:18.821269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125f29e-12e9-4be2-bdc5-0bc0108f87fc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:18.825026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c354efc-e104-4949-b52d-0780260fa0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:18.825673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c354efc-e104-4949-b52d-0780260fa0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:18.825960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c354efc-e104-4949-b52d-0780260fa0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:18.826239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c354efc-e104-4949-b52d-0780260fa0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:18.835548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c354efc-e104-4949-b52d-0780260fa0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:18.837016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c354efc-e104-4949-b52d-0780260fa0b6 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:18.846826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1107/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:19.177165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.178657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:19.190602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd0a16-1914-44f1-84a7-609bd9bdafa2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.196746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.197717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.198007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.198295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] PATH_INFO: `/v3/projects/22eee074e0a44037a063390ff9f47f64` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.199647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22eee074e0a44037a063390ff9f47f64)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.201602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.263849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22eee074e0a44037a063390ff9f47f64', 'request_id': 'req-8bee6bd3-34cf-4868-9846-ba7e96111977'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:19.269588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22eee074e0a44037a063390ff9f47f64', 'request_id': 'req-8bee6bd3-34cf-4868-9846-ba7e96111977'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:19.277709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22eee074e0a44037a063390ff9f47f64', 'request_id': 'req-8bee6bd3-34cf-4868-9846-ba7e96111977'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:19.290180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] The token cache is being invalidate because project 22eee074e0a44037a063390ff9f47f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:19.290501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:19.291166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8bee6bd3-34cf-4868-9846-ba7e96111977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:19.292862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2191] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:19 2022] DELETE /identity/v3/projects/22eee074e0a44037a063390ff9f47f64 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:19.298318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.298613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.298903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.305381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:19.320655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:19.322079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:19.395336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1108/2192] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:19.397425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.397841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.398107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.417521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:19.418708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:19.427194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2193] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:19.431397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.441675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f9298-951e-40a6-a17c-05fdd370e7a6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.446085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.446763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.447045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.447319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.448039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:19.448376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.449972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.461499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1109/2194] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:19.465618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.475402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0ecb9-9048-48ef-8c17-e80e8a900ede None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.480075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.480752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.481039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.481318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.482010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.483897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.510234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2195] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:19 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:19.514462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.525251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c60ac-4f6d-4497-a3f0-1309c2d86f90 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.584410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.585085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.585453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.585624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.586325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.588356 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.611177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1110/2196] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:19 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:19.612499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.623806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacb181-9d2e-4d81-8178-ac036c21f403 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.629032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.629847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.630138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.630442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.631521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.633387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.646983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2197] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:19.651028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.661456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6f16e-1789-4d30-b9c7-ea1feec7436f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.666091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30b06e2-5158-47b3-aae1-8bf9eecb1c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.666752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b06e2-5158-47b3-aae1-8bf9eecb1c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.667040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b06e2-5158-47b3-aae1-8bf9eecb1c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.667323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b06e2-5158-47b3-aae1-8bf9eecb1c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.668053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b06e2-5158-47b3-aae1-8bf9eecb1c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.669692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30b06e2-5158-47b3-aae1-8bf9eecb1c8d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.680153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1111/2198] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:19.684110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.694650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f4ba6-fec8-4282-bf7f-e95e3d4819b1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.699114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.699983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.700303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.700584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] PATH_INFO: `/v3/projects/e9181ed0a5c14a54b7d15417e147ee11/users/c9440928fa144c25a9f959e28cd6e866/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.712773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:19.716146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9181ed0a5c14a54b7d15417e147ee11, user_id=c9440928fa144c25a9f959e28cd6e866, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.719542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.732503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2199] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:19 2022] PUT /identity/v3/projects/e9181ed0a5c14a54b7d15417e147ee11/users/c9440928fa144c25a9f959e28cd6e866/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:19.744107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf07e0-ef23-4a2f-a706-9a2cf61e5f00 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.744445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf07e0-ef23-4a2f-a706-9a2cf61e5f00 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.744732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf07e0-ef23-4a2f-a706-9a2cf61e5f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.771734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5ecf07e0-ef23-4a2f-a706-9a2cf61e5f00 None None] MFA Rules not processed for user `c9440928fa144c25a9f959e28cd6e866`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:19.773013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5ecf07e0-ef23-4a2f-a706-9a2cf61e5f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:19.833334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:19.842875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1112/2200] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:19.850804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:19.852798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:19.912377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce122eb-c961-438a-9a6f-4c412882da67 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:19.915950 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7c4d3-7086-4720-9a52-b96b69139c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:19.916622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7c4d3-7086-4720-9a52-b96b69139c7b service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:19.916904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7c4d3-7086-4720-9a52-b96b69139c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:19.917181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7c4d3-7086-4720-9a52-b96b69139c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:19.925548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7c4d3-7086-4720-9a52-b96b69139c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:19.926966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7c4d3-7086-4720-9a52-b96b69139c7b service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:19.935635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2201] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:21.267675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b527-f54e-419d-8b16-025e09a801e3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:21.268202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b527-f54e-419d-8b16-025e09a801e3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:21.268327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704b527-f54e-419d-8b16-025e09a801e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:21.286531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c704b527-f54e-419d-8b16-025e09a801e3 None None] MFA Rules not processed for user `c9440928fa144c25a9f959e28cd6e866`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:21.287740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c704b527-f54e-419d-8b16-025e09a801e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:21.294758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1113/2202] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:27.082416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdd136-55d5-44de-a9e1-ab586d7df4e3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:27.082941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdd136-55d5-44de-a9e1-ab586d7df4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:27.083024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cdd136-55d5-44de-a9e1-ab586d7df4e3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:27.084364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2203] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 12:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 12:32:27.119053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839e66-ca53-4650-a673-515109a80502 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:27.119596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839e66-ca53-4650-a673-515109a80502 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:27.119596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1839e66-ca53-4650-a673-515109a80502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:27.148983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e1839e66-ca53-4650-a673-515109a80502 None None] MFA Rules not processed for user `06d3500f0bda4ec58b2d561f13eb42d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:27.150585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e1839e66-ca53-4650-a673-515109a80502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:27.203020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdd136-55d5-44de-a9e1-ab586d7df4e3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:27.205028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64cdd136-55d5-44de-a9e1-ab586d7df4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:27.243543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1114/2204] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:27.262717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1839e66-ca53-4650-a673-515109a80502 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:27.264274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e1839e66-ca53-4650-a673-515109a80502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:27.298251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1839e66-ca53-4650-a673-515109a80502 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:27.316234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cdd136-55d5-44de-a9e1-ab586d7df4e3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:27.323009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:27.323980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:27.324518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:27.325021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:27.326569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:27.328108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:27.328483 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:27.328777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] PATH_INFO: `/v3/users/779f59f5eb434a758ee317f72da17fd5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:27.330074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779f59f5eb434a758ee317f72da17fd5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:27.332299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:27.345328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:27.347036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:27.364564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1115/2205] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:27.409935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779f59f5eb434a758ee317f72da17fd5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.410698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779f59f5eb434a758ee317f72da17fd5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.420422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779f59f5eb434a758ee317f72da17fd5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.428688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779f59f5eb434a758ee317f72da17fd5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.445611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c31a8bb6-2b60-4f88-af08-f04b7a8ee4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779f59f5eb434a758ee317f72da17fd5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.453130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2206] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:27 2022] DELETE /identity/v3/users/779f59f5eb434a758ee317f72da17fd5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:27.709472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:27.729227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7b704-f490-4567-9f59-d5c39eca3adb service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:27.848555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:27.849788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:27.849788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:27.850186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] PATH_INFO: `/v3/projects/2073f419769a42be967d132787f9ad1e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:27.851530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2073f419769a42be967d132787f9ad1e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:27.853556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:27.914559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2073f419769a42be967d132787f9ad1e', 'request_id': 'req-0595d73e-dff7-44de-9f66-dbff9166985d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.927773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2073f419769a42be967d132787f9ad1e', 'request_id': 'req-0595d73e-dff7-44de-9f66-dbff9166985d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.942157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2073f419769a42be967d132787f9ad1e', 'request_id': 'req-0595d73e-dff7-44de-9f66-dbff9166985d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.954703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] The token cache is being invalidate because project 2073f419769a42be967d132787f9ad1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:27.954993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.955809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0595d73e-dff7-44de-9f66-dbff9166985d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:27.957510 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1116/2207] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:27 2022] DELETE /identity/v3/projects/2073f419769a42be967d132787f9ad1e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:27.966742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:27.967089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:27.967654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.055115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:28.056959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:28.071288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:28.204926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2208] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:28.213517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.214441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.214941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.235046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:28.242237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:28.243701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:28.259862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1117/2209] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:28.264051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.278095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9cf61-a0d6-4027-8321-d817779ca11d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:28.284838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:28.294279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.294803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.295286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.296529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:28.297205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:28.299081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:28.325768 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2210] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:28.326794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.342565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dcc37-5bd7-4ef4-8f85-d6160dad6c50 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:28.357818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:28.359530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.359942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.360430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.361972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:28.366766 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:28.424559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1118/2211] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:28 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:28.426882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.441713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb37c93-8a59-4b46-98f1-809387f9587b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:28.551655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:28.552554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.552847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.553132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.553838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:28.555890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:28.577354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2212] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:28 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:28.582093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.608698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920dc22-f63b-4f16-9dae-2a53f242eab7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:28.617029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:28.617843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.618421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.618763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.621143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:28.622765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:28.644753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1119/2213] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:28.648890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.677531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28741109-ce89-4e2c-bfff-44e5534819c0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:28.683047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085dc7e-974b-4d79-85c0-aa4cd3d54aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:28.683965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dc7e-974b-4d79-85c0-aa4cd3d54aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.684455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dc7e-974b-4d79-85c0-aa4cd3d54aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.684991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085dc7e-974b-4d79-85c0-aa4cd3d54aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.685883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dc7e-974b-4d79-85c0-aa4cd3d54aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:28.688068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085dc7e-974b-4d79-85c0-aa4cd3d54aa5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:28.700484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2214] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:28.705260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.717975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab614b2-99b6-40b3-9708-09d191a0c3db admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:28.737768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:28.740340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.740643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.740943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] PATH_INFO: `/v3/projects/3528f0d517e54016ab254ca0a3c11693/users/1a1284271e83435aa6fd851d6106e3a8/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.764082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3528f0d517e54016ab254ca0a3c11693, user_id=1a1284271e83435aa6fd851d6106e3a8, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:28.768086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:28.786277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1120/2215] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:28 2022] PUT /identity/v3/projects/3528f0d517e54016ab254ca0a3c11693/users/1a1284271e83435aa6fd851d6106e3a8/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:28.791645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22f410-52b0-4ad7-a857-73873527ad4b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:28.791877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22f410-52b0-4ad7-a857-73873527ad4b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:28.792169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22f410-52b0-4ad7-a857-73873527ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:28.824721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1f22f410-52b0-4ad7-a857-73873527ad4b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:28.826220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f22f410-52b0-4ad7-a857-73873527ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:28.926717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2216] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:28.942946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:28.945346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:29.048141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82cae3-8170-4d20-bd97-157097e4e868 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:29.054329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:29.055023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:29.055284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:29.055785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:29.096234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:29.096929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:29.122587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1121/2217] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:34.632236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf1005-6f14-4b19-bc95-d69be3295039 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:34.632236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf1005-6f14-4b19-bc95-d69be3295039 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:34.632866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf1005-6f14-4b19-bc95-d69be3295039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:34.672266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cdaf1005-6f14-4b19-bc95-d69be3295039 None None] MFA Rules not processed for user `1a1284271e83435aa6fd851d6106e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:34.673624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cdaf1005-6f14-4b19-bc95-d69be3295039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:34.682923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:34.764106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2218] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:34.777933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:34.779819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:34.972850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b850c99-bfd4-4eba-82c3-10b43a501bad service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:34.980647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:34.981480 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:34.981907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:34.982426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:35.008472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:35.010423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:35.029341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1122/2219] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:37.395101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:37.395551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:37.395713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:37.424237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] MFA Rules not processed for user `1a1284271e83435aa6fd851d6106e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:37.425425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:37.452213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2220] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:44.707944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:44.709949 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:44.733035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218996c-1703-4f63-8e96-ef5491c9e554 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:44.738326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:44.739297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:44.739588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:44.739871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] PATH_INFO: `/v3/users/4626b73e62fc44cda6e1c5da39e81752` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:44.741554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4626b73e62fc44cda6e1c5da39e81752)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:44.743496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:44.772762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:44.798059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4626b73e62fc44cda6e1c5da39e81752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:44.805880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4626b73e62fc44cda6e1c5da39e81752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:44.806524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4626b73e62fc44cda6e1c5da39e81752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:44.819182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4626b73e62fc44cda6e1c5da39e81752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:44.825985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4626b73e62fc44cda6e1c5da39e81752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:44.832524 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1123/2221] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:44 2022] DELETE /identity/v3/users/4626b73e62fc44cda6e1c5da39e81752 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:45.017765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.032432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229ca5e-5c67-4b42-8731-656ce284f211 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:45.089034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:45.089894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.090178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.090454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] PATH_INFO: `/v3/projects/0a3fe8942a3b42bdbc8e5a1c266faacc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.091649 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3fe8942a3b42bdbc8e5a1c266faacc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:45.093631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:45.143927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3fe8942a3b42bdbc8e5a1c266faacc', 'request_id': 'req-64090d49-7dc1-48e9-8425-6b3e8efb3171'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.151323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3fe8942a3b42bdbc8e5a1c266faacc', 'request_id': 'req-64090d49-7dc1-48e9-8425-6b3e8efb3171'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.163084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3fe8942a3b42bdbc8e5a1c266faacc', 'request_id': 'req-64090d49-7dc1-48e9-8425-6b3e8efb3171'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.181394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] The token cache is being invalidate because project 0a3fe8942a3b42bdbc8e5a1c266faacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:45.181684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.182385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.184249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2222] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:45 2022] DELETE /identity/v3/projects/0a3fe8942a3b42bdbc8e5a1c266faacc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:45.188958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.192945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:45.208536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254605-07f6-4e72-84f1-0c7be378df9c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:45.271301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:45.274293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.274293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.274293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] PATH_INFO: `/v3/users/1e2c9c4fda4447d898a2d0aafb6c9959` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.274293 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2c9c4fda4447d898a2d0aafb6c9959)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:45.276054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:45.326242 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2c9c4fda4447d898a2d0aafb6c9959'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.334020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2c9c4fda4447d898a2d0aafb6c9959'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.334806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2c9c4fda4447d898a2d0aafb6c9959'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.349358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2c9c4fda4447d898a2d0aafb6c9959'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.356488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-dcf70ca9-282c-42fd-bbe5-d27e84c14342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2c9c4fda4447d898a2d0aafb6c9959'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.363693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1124/2223] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:45 2022] DELETE /identity/v3/users/1e2c9c4fda4447d898a2d0aafb6c9959 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:45.558751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.560171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:45.588940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64090d49-7dc1-48e9-8425-6b3e8efb3171 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:45.638167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:45.667691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:45.668696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.668986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.669266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] PATH_INFO: `/v3/projects/3bde225a82cd49519ed903a76147ddfe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.670548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bde225a82cd49519ed903a76147ddfe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:45.672794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:45.723612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bde225a82cd49519ed903a76147ddfe', 'request_id': 'req-3c01e705-336f-49d1-9f7a-93d473022767'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.729684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bde225a82cd49519ed903a76147ddfe', 'request_id': 'req-3c01e705-336f-49d1-9f7a-93d473022767'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.736589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bde225a82cd49519ed903a76147ddfe', 'request_id': 'req-3c01e705-336f-49d1-9f7a-93d473022767'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.748154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] The token cache is being invalidate because project 3bde225a82cd49519ed903a76147ddfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:45.748476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.749137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.750923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2224] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:45 2022] DELETE /identity/v3/projects/3bde225a82cd49519ed903a76147ddfe => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:45.756315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af3465-58de-4041-b7f3-104352237e6a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.756776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af3465-58de-4041-b7f3-104352237e6a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.756872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af3465-58de-4041-b7f3-104352237e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.776336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84af3465-58de-4041-b7f3-104352237e6a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:45.777597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84af3465-58de-4041-b7f3-104352237e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:45.791556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:45.848314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1125/2225] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:45.848511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.848511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:45.849734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.850026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.850308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.857440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01e705-336f-49d1-9f7a-93d473022767 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:45.862806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:45.863836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.865059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.865363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] PATH_INFO: `/v3/users/87de392663e541fd87dd6a94288d3477` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.866831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87de392663e541fd87dd6a94288d3477)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:45.868889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:45.869938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:45.871222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:45.880051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1126/2226] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:45.884673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.893916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834d0d3-9bb2-49e8-ab51-0e0933975b6c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:45.898599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:45.899251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.899537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.899818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.900709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:45.901097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:45.902714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:45.915191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1127/2227] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:45.919811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.923977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87de392663e541fd87dd6a94288d3477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.925701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87de392663e541fd87dd6a94288d3477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.929957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a289a-ef25-425d-b662-2c587c93e8e9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:45.934673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87de392663e541fd87dd6a94288d3477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.934885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d5630-7713-406c-8883-b8fc329453fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:45.935438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d5630-7713-406c-8883-b8fc329453fb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.935724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d5630-7713-406c-8883-b8fc329453fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.936002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d5630-7713-406c-8883-b8fc329453fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:45.937281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d5630-7713-406c-8883-b8fc329453fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:45.939184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d5630-7713-406c-8883-b8fc329453fb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:45.943489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87de392663e541fd87dd6a94288d3477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.956853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87de392663e541fd87dd6a94288d3477'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:45.963917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2228] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:45 2022] DELETE /identity/v3/users/87de392663e541fd87dd6a94288d3477 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:45.969326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1128/2229] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:45 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:45.970432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:45.971288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:45.971581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:45.971826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:45.971946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.014748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:46.017239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:46.035500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:46.069882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ff32-978f-4c5a-b5ef-1543db1e9423 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.082338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1129/2230] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:46.091828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.093871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:46.133548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.134389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.134799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.135218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.136128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.138568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.162502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d91963-ff98-4eff-b800-522a208cdd55 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.168189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2231] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:45 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:46.169324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.169890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.170721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.171037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.171364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.183539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5664-1fc3-497b-ad17-bde15cd7636a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.184505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.186954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.190386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.191258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.191554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.191955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.194493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.196126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.203677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1130/2232] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:46.212390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2233] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:46.215770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.228040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2154dd-f55c-4cc6-9bbb-6345a19a95ec service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.233973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f1ea-cdc5-4c9d-8f6a-5a00de61776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.234755 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f1ea-cdc5-4c9d-8f6a-5a00de61776d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.235055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f1ea-cdc5-4c9d-8f6a-5a00de61776d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.235338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f1ea-cdc5-4c9d-8f6a-5a00de61776d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.236411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f1ea-cdc5-4c9d-8f6a-5a00de61776d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.237995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f1ea-cdc5-4c9d-8f6a-5a00de61776d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.252300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1131/2234] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:46.254795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.267382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650874a-6707-4833-bf70-a4f17a0aee68 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.274085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.276574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.277101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.277588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] PATH_INFO: `/v3/projects/6f6b44bed4904d50b5f1ebfc5ab34743/users/70c035662359493e86c58a3d40150eb8/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.308321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6b44bed4904d50b5f1ebfc5ab34743, user_id=70c035662359493e86c58a3d40150eb8, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.312057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.331073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2235] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:46 2022] PUT /identity/v3/projects/6f6b44bed4904d50b5f1ebfc5ab34743/users/70c035662359493e86c58a3d40150eb8/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:46.336514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.336806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.337087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.361680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:46.363553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:46.369787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:46.447979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1132/2236] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:46.469770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.555956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2149cb0-d99a-4867-9cac-03453fb27b8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.561202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.561202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.561202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.561202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.572824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.574658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.590076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2237] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:46.712501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.715333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:46.730558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf036ee-f23e-45d9-88ab-13142cb00ede None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.737935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:46.738918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:46.739336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:46.740165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] PATH_INFO: `/v3/projects/0f705d4065054c75b90eaaaf6d77e275` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:46.741138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f705d4065054c75b90eaaaf6d77e275)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:46.743717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:46.812153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f705d4065054c75b90eaaaf6d77e275', 'request_id': 'req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:46.822590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f705d4065054c75b90eaaaf6d77e275', 'request_id': 'req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:46.832335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f705d4065054c75b90eaaaf6d77e275', 'request_id': 'req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:46.847948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] The token cache is being invalidate because project 0f705d4065054c75b90eaaaf6d77e275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:46.848537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:46.849798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:46.851678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1133/2238] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:46 2022] DELETE /identity/v3/projects/0f705d4065054c75b90eaaaf6d77e275 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:46.856907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:46.872566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94448c4e-0b32-4c5e-8e27-77af1549f90a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:46.965076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:47.021183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:47.022728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:47.023022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:47.024079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] PATH_INFO: `/v3/users/78edec2b73164de48d4ccd3926651010` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:47.025501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78edec2b73164de48d4ccd3926651010)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:47.028707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:47.140558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78edec2b73164de48d4ccd3926651010'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.141911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78edec2b73164de48d4ccd3926651010'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.154256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78edec2b73164de48d4ccd3926651010'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.174935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78edec2b73164de48d4ccd3926651010'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.195163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78edec2b73164de48d4ccd3926651010'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.212405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2239] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:46 2022] DELETE /identity/v3/users/78edec2b73164de48d4ccd3926651010 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:47.649229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:47.651350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:47.684746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25262a8-2d05-4ab9-9f53-422ea5a4c8da admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:47.768000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:47.768940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:47.769241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:47.769511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] PATH_INFO: `/v3/projects/05016b1c2cfb4e00a61ba5ab8d9b756d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:47.770742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05016b1c2cfb4e00a61ba5ab8d9b756d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:47.772902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:47.816604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:47.818452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:47.836434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf721b9-c503-47ff-a5ae-2d5d920d11d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:47.839382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:47.855543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05016b1c2cfb4e00a61ba5ab8d9b756d', 'request_id': 'req-8dc88f1c-70ab-484b-b33f-dcddc0f52623'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.869364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05016b1c2cfb4e00a61ba5ab8d9b756d', 'request_id': 'req-8dc88f1c-70ab-484b-b33f-dcddc0f52623'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.888295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05016b1c2cfb4e00a61ba5ab8d9b756d', 'request_id': 'req-8dc88f1c-70ab-484b-b33f-dcddc0f52623'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.905976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] The token cache is being invalidate because project 05016b1c2cfb4e00a61ba5ab8d9b756d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:47.907243 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.908389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8dc88f1c-70ab-484b-b33f-dcddc0f52623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:47.916432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1134/2240] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:47 2022] DELETE /identity/v3/projects/05016b1c2cfb4e00a61ba5ab8d9b756d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:47.918397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8ca18-6220-4283-81a4-b160a7d8e466 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:47.919170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8ca18-6220-4283-81a4-b160a7d8e466 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:47.919677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8ca18-6220-4283-81a4-b160a7d8e466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:47.934554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:47.937067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:47.937536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:47.938015 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] PATH_INFO: `/v3/users/5e1db2b5cde545c993d6b8a5c5458d3a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:47.941027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e1db2b5cde545c993d6b8a5c5458d3a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:47.943999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:47.946846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-21c8ca18-6220-4283-81a4-b160a7d8e466 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:47.949156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-21c8ca18-6220-4283-81a4-b160a7d8e466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:47.961375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1135/2241] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:47.970129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:47.970421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:47.970702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:47.990470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:47.991770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.001537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1136/2242] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:48.007404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.010184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e1db2b5cde545c993d6b8a5c5458d3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.010922 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e1db2b5cde545c993d6b8a5c5458d3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.019670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b6dd2-9d4a-4cae-b940-e3c767033f84 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.021383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e1db2b5cde545c993d6b8a5c5458d3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.025401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.026322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.026629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.026881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.027643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:48.027966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.028376 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:48.030622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.031042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e1db2b5cde545c993d6b8a5c5458d3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.043526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1137/2243] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:48.049664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.051521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.053474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e077fab4-7636-4978-988f-1d8e3709179b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e1db2b5cde545c993d6b8a5c5458d3a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.062568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2244] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:47 2022] DELETE /identity/v3/users/5e1db2b5cde545c993d6b8a5c5458d3a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:48.067165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.067445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.067744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.068899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c37cd2-7c96-4d64-b145-0e4a4d854af8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.087384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:48.089082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.134432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.135106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.135394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.135672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.136396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.138317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.166593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2245] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:48.171446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1138/2246] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:48 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:48.175634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.181329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.184013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.186255 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282535e-aea1-4fe1-974b-15448689c7d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.262474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.263166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.264061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.264430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.265156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.265646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38315312-e999-4060-aee1-1f19c55fc7a0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.267465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.269899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.270639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.270996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.271394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.284007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.285577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.312959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1139/2247] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:48.314381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.315661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2248] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:48 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:48.316998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.321111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.338786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbc387-2f33-499c-b24e-43ae294f6c6a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.342116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee365b-6849-4747-9f18-c2add150416f service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.346315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.347413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.347840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.348143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.348937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.350504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.350896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.352013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.352518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.352985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] PATH_INFO: `/v3/users/dd95c260fea24556b15707fde82b6833` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.362221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd95c260fea24556b15707fde82b6833)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.365537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.384654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2249] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:48.385646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.418454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c021832-b12a-4613-a774-4d57b983b53b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.429624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:48.433874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca514e-442f-402b-8a4f-4bee36073a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.434629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca514e-442f-402b-8a4f-4bee36073a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.435069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca514e-442f-402b-8a4f-4bee36073a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.435352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca514e-442f-402b-8a4f-4bee36073a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.436132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca514e-442f-402b-8a4f-4bee36073a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.437751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca514e-442f-402b-8a4f-4bee36073a42 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.480265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2250] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:48.481011 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909395e-db59-4453-8a3d-fd335d4e7a61 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.481300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909395e-db59-4453-8a3d-fd335d4e7a61 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.481678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909395e-db59-4453-8a3d-fd335d4e7a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.487162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd95c260fea24556b15707fde82b6833'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.500864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd95c260fea24556b15707fde82b6833'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.501873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd95c260fea24556b15707fde82b6833'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.520725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd95c260fea24556b15707fde82b6833'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.539758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd95c260fea24556b15707fde82b6833'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:48.548045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1140/2251] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:48 2022] DELETE /identity/v3/users/dd95c260fea24556b15707fde82b6833 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:48.548947 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.550395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.586247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1909395e-db59-4453-8a3d-fd335d4e7a61 None None] MFA Rules not processed for user `70c035662359493e86c58a3d40150eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:48.587826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1909395e-db59-4453-8a3d-fd335d4e7a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.683067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:48.721233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f49114-1535-473a-9486-9e53391daa94 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.726746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.728059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.728541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.728987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] PATH_INFO: `/v3/projects/8608c1c9151d489997547a678e73abbf/users/bb9a064a60374cfcbfee11fc1775e752/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.763056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8608c1c9151d489997547a678e73abbf, user_id=bb9a064a60374cfcbfee11fc1775e752, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:48.767283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:48.791881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2252] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:48.791881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b7405-c4d9-429a-a8cd-329076d8eab9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.791881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b7405-c4d9-429a-a8cd-329076d8eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.801012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b7405-c4d9-429a-a8cd-329076d8eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.804741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1141/2253] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:48 2022] PUT /identity/v3/projects/8608c1c9151d489997547a678e73abbf/users/bb9a064a60374cfcbfee11fc1775e752/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:48.806014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:48.808120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.843333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd4b7405-c4d9-429a-a8cd-329076d8eab9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:48.844680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd4b7405-c4d9-429a-a8cd-329076d8eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:48.938540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450298b-a122-4906-8593-44c904f76953 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:48.943050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:48.943953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.944352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.944713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:48.994199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2254] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:48.995655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:48.995954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:48.996259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.067553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] MFA Rules not processed for user `bb9a064a60374cfcbfee11fc1775e752`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:49.068895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:49.082288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:49.134550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.137168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.151423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1142/2255] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:49.152349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.153784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:49.206708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2256] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:49.207467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.215642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:49.250988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417cdb7-ee78-4cbc-a3a5-b7fd96d7b548 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.264690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17645ac2-4e14-4913-a6ea-0bb27a301c74 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.265264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.266920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.267216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.267499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] PATH_INFO: `/v3/projects/134292b9fbb94480a7bcaa7509d1e283` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.276499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=134292b9fbb94480a7bcaa7509d1e283)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.280598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.315080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.315756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.316057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.316368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.345458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.346924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.378994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1143/2257] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:49.381678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.393285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134292b9fbb94480a7bcaa7509d1e283', 'request_id': 'req-66eff3a7-dd70-4d08-813a-9b0d4916ad02'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:49.408112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134292b9fbb94480a7bcaa7509d1e283', 'request_id': 'req-66eff3a7-dd70-4d08-813a-9b0d4916ad02'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:49.412419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:49.421342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134292b9fbb94480a7bcaa7509d1e283', 'request_id': 'req-66eff3a7-dd70-4d08-813a-9b0d4916ad02'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:49.426362 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ab58d-9cc3-4c5e-ab5d-7b622fe64de0 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.455178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] The token cache is being invalidate because project 134292b9fbb94480a7bcaa7509d1e283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:49.455540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:49.458781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-66eff3a7-dd70-4d08-813a-9b0d4916ad02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:49.464070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2258] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:49 2022] DELETE /identity/v3/projects/134292b9fbb94480a7bcaa7509d1e283 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:49.471261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0122-81c6-43a3-85ab-d00c5194b2f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.471562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0122-81c6-43a3-85ab-d00c5194b2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.471849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c0122-81c6-43a3-85ab-d00c5194b2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.513678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-724c0122-81c6-43a3-85ab-d00c5194b2f5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:49.531322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-724c0122-81c6-43a3-85ab-d00c5194b2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:49.554959 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.558343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.559441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.560973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.565096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.569545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.657299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1144/2259] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:49 2022] POST /identity/v3/projects => generated 387 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:49.658970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.668498 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:49.690855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2260] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:49.702825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.703563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.704204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.754748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:49.763239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:49.811321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb8542-9db2-4438-86a1-3f7c1ad0e78e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.816760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.817521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.817876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.818222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.819091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.821371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.831063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:49.846082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1145/2261] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:49 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:49.851508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.858773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2262] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:49.862552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a267cc-a007-4223-a459-6291448ec9f4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.863002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.867367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.868072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.868394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.869516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.869516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.871088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.874289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302776c7-22de-4c56-a084-7b8c5b5e1384 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.880482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.881857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.882288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.882708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.883726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:49.883858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1146/2263] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:49.884338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.886378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.891346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.903357 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2264] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:32:49.908197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.910791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0a7e-4b91-4fb4-9991-7dcf39430a1c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.918688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.919986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.920474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.920973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] PATH_INFO: `/v3/projects/4f59a894cc964acaac7fabe8821cdc46/users/8c0aa23266a04320bedc88e63d57777e/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.921760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5345f6-07b5-4536-a753-f0de1601706c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:49.928946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:49.929850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:49.930436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:49.930872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:49.933186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.937240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.943884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f59a894cc964acaac7fabe8821cdc46, user_id=8c0aa23266a04320bedc88e63d57777e, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:49.949185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:49.975513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1147/2265] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:49 2022] PUT /identity/v3/projects/4f59a894cc964acaac7fabe8821cdc46/users/8c0aa23266a04320bedc88e63d57777e/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:49.979495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:49.995710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2266] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:49 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:32:49.996810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.083740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:50.090372 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f8b62-1ab5-4221-885a-9c66cd8b0b50 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.097248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.098349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.098799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.100166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.101147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.103579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.114122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a22fe4-6ef7-4247-8289-fc62e085f34b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.122009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.122772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.123167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.125219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.126737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.131768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.136416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2267] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:49 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:50.140817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.152898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a689d51-d27a-42fa-a0be-9889f40fd547 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.156275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1148/2268] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:50.160725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.167832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.169448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.169448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.169448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.170165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.172213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.179256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2608f-bf15-4fe8-b11b-3d944e9ec168 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.186571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d3903-ced5-4571-83e4-5adbfe4c1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.187541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3903-ced5-4571-83e4-5adbfe4c1025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.188099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3903-ced5-4571-83e4-5adbfe4c1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.188833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d3903-ced5-4571-83e4-5adbfe4c1025 admin admin] PATH_INFO: `/v3/projects/4f59a894cc964acaac7fabe8821cdc46/users/8c0aa23266a04320bedc88e63d57777e/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.191863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d3903-ced5-4571-83e4-5adbfe4c1025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f59a894cc964acaac7fabe8821cdc46, user_id=8c0aa23266a04320bedc88e63d57777e, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.197485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d3903-ced5-4571-83e4-5adbfe4c1025 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.198871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2269] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:50.200197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.214769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1149/2270] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:50 2022] PUT /identity/v3/projects/4f59a894cc964acaac7fabe8821cdc46/users/8c0aa23266a04320bedc88e63d57777e/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:50.225575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.225887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.226152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.230466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bbf3f-6fd5-40b2-9d47-abc87c7338c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.269125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] MFA Rules not processed for user `8c0aa23266a04320bedc88e63d57777e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:50.271749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.350277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10de61-5077-4a2e-9a12-616eb2ca6071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.351130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10de61-5077-4a2e-9a12-616eb2ca6071 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.352807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10de61-5077-4a2e-9a12-616eb2ca6071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.353105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10de61-5077-4a2e-9a12-616eb2ca6071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.353864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10de61-5077-4a2e-9a12-616eb2ca6071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.355536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10de61-5077-4a2e-9a12-616eb2ca6071 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.371064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2271] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:50.372445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.372741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.373080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.396003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] MFA Rules not processed for user `f832895e13784dfaa69b44505da8855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:50.398822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.422760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1150/2272] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:50.424304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.440221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1858ba-e53f-4768-9c3f-d04d45118f8d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.446367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cec4b-6135-48bf-90aa-8d8ac2cfb182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.447449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cec4b-6135-48bf-90aa-8d8ac2cfb182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.447946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cec4b-6135-48bf-90aa-8d8ac2cfb182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.448263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cec4b-6135-48bf-90aa-8d8ac2cfb182 admin admin] PATH_INFO: `/v3/projects/b4a603ce9a934f188a137ee171d66aa7/users/880c0417a58d48ceaab3912957f2e4a7/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.473168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cec4b-6135-48bf-90aa-8d8ac2cfb182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4a603ce9a934f188a137ee171d66aa7, user_id=880c0417a58d48ceaab3912957f2e4a7, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.486451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cec4b-6135-48bf-90aa-8d8ac2cfb182 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.490050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2273] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:50.491993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.494601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.506561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1151/2274] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:50 2022] PUT /identity/v3/projects/b4a603ce9a934f188a137ee171d66aa7/users/880c0417a58d48ceaab3912957f2e4a7/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:50.512107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.512700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.513144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.523659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:50.539736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:50.541636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.609196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3403aef-b537-4adb-891a-ac0faa83e00c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.699539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1152/2275] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:50.710789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.712272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.714695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:50.777413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.778274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.778647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.779008 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.857865 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69382c58-74b8-44bf-bdcf-25a84b416f3d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:50.862475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de62ba1-1024-441c-8e1a-c6c51d839f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:50.863145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62ba1-1024-441c-8e1a-c6c51d839f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.863430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62ba1-1024-441c-8e1a-c6c51d839f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.863707 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de62ba1-1024-441c-8e1a-c6c51d839f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.886599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62ba1-1024-441c-8e1a-c6c51d839f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.890499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de62ba1-1024-441c-8e1a-c6c51d839f45 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.913690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1153/2276] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:50.918071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:50.919964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:50.920644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:50.920936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:50.922665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:50.949815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2277] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:50.952840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:50.956400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.961964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] MFA Rules not processed for user `70c035662359493e86c58a3d40150eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:50.964254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:50.989358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e0207-65aa-41bf-9bac-c8527545051f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.006137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.007258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.007751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.008144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] PATH_INFO: `/v3/projects/5eba094c78f14adfbade06ea4f93065e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.011613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eba094c78f14adfbade06ea4f93065e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.020462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.141548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1154/2278] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:51.150803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.164746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eba094c78f14adfbade06ea4f93065e', 'request_id': 'req-c19dec62-06c4-4aac-8455-c59f2f6c8871'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.169857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23860fa-3d43-4b94-87be-2c5e751846d2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.175946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eba094c78f14adfbade06ea4f93065e', 'request_id': 'req-c19dec62-06c4-4aac-8455-c59f2f6c8871'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.189310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eba094c78f14adfbade06ea4f93065e', 'request_id': 'req-c19dec62-06c4-4aac-8455-c59f2f6c8871'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.213178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] The token cache is being invalidate because project 5eba094c78f14adfbade06ea4f93065e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:51.213743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.214611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c19dec62-06c4-4aac-8455-c59f2f6c8871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.220494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2279] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:50 2022] DELETE /identity/v3/projects/5eba094c78f14adfbade06ea4f93065e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:51.227486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782138-0e60-4ca0-91ca-74cbe185b373 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.229368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782138-0e60-4ca0-91ca-74cbe185b373 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.229683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd782138-0e60-4ca0-91ca-74cbe185b373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.261809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bd782138-0e60-4ca0-91ca-74cbe185b373 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:51.263558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd782138-0e60-4ca0-91ca-74cbe185b373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:51.271816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:51.290851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.292034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.293226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.293645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] PATH_INFO: `/v3/users/bb9a064a60374cfcbfee11fc1775e752` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.297845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9a064a60374cfcbfee11fc1775e752)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.299923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.363046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2280] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:51.372236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.372566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.373087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.383156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:51.397856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:51.400286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:51.417327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2281] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:51.418491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.428348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9a064a60374cfcbfee11fc1775e752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.429849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5539c71-0216-4692-a406-a067cc30c2a1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.450380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9a064a60374cfcbfee11fc1775e752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.451207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.451716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9a064a60374cfcbfee11fc1775e752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.452504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.454172 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.454526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.456061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:51.457097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.462126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.488127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9a064a60374cfcbfee11fc1775e752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.503724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2282] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:51.505585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.518149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:51.518283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-29c0845a-21dd-49e8-ad7d-b70033a1ba48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9a064a60374cfcbfee11fc1775e752'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:51.525923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1155/2283] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:51 2022] DELETE /identity/v3/users/bb9a064a60374cfcbfee11fc1775e752 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:51.530956 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.532561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.533089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.540680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32526ad0-a7b8-45d7-8099-7fad36464e4a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.590252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:51.594608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:51.720048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.723007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.724731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.725168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.726017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.728193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.780325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1156/2284] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:51.782624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2285] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:51 2022] POST /identity/v3/projects => generated 385 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:51.784588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.795916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd4c8c-ef1f-4f62-8768-2fd91f5ec928 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.810409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.814430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:51.871948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.872908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.874285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.874985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.876931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.879218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.916948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:51.926062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1157/2286] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:51 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:51.929051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.942311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc279221-0858-403d-9bdd-352015fe0167 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.946725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11e92-feeb-4894-9307-3147c0c749f5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.952361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.953138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.953435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.953718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.954129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.954509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.955128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.955518 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.956136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.956577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.968292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1158/2287] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:51.970804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:51.980718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.982467 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.983418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319c6f9-2b40-4493-8f06-9d30b9ef4903 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:51.988517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24bd94-dd51-4a56-9fd8-7919d3f4f803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:51.989182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bd94-dd51-4a56-9fd8-7919d3f4f803 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:51.989467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bd94-dd51-4a56-9fd8-7919d3f4f803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:51.990397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24bd94-dd51-4a56-9fd8-7919d3f4f803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:51.991211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24bd94-dd51-4a56-9fd8-7919d3f4f803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:51.993087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24bd94-dd51-4a56-9fd8-7919d3f4f803 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:51.998266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:32:52.009547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1159/2289] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:52.011132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:52.022700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee813a66-324d-4374-9dc1-522a312705c9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:52.027784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:52.029021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:52.029170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:52.029604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] PATH_INFO: `/v3/projects/c3ff2fe9ee5d4784bdb104a2e750f8e6/users/1f34d29c3fa647d89dfce583e1d40705/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:52.048259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff2fe9ee5d4784bdb104a2e750f8e6, user_id=1f34d29c3fa647d89dfce583e1d40705, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:52.052109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:52.077368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2290] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:52 2022] PUT /identity/v3/projects/c3ff2fe9ee5d4784bdb104a2e750f8e6/users/1f34d29c3fa647d89dfce583e1d40705/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:52.088233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:52.088678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:52.089120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:52.125741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] MFA Rules not processed for user `1f34d29c3fa647d89dfce583e1d40705`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:52.127499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:52.235912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1160/2291] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:52.250225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:52.250225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:52.413810 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:52.446645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a1173-4244-4b37-b387-7e2528f1b363 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:52.457311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ff5d1-8252-4cb2-99e5-b51b06aa964c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:52.458084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ff5d1-8252-4cb2-99e5-b51b06aa964c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:52.458380 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ff5d1-8252-4cb2-99e5-b51b06aa964c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:52.458662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ff5d1-8252-4cb2-99e5-b51b06aa964c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:52.464460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:52.466127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:52.482034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ff5d1-8252-4cb2-99e5-b51b06aa964c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:52.483594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ff5d1-8252-4cb2-99e5-b51b06aa964c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:52.507788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2292] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:52.509482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:52.509778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:52.510060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:52.557432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:52.561207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:52.663442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52844d7-26ef-40e0-be4f-d27a9e5dfee5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:52.666618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2293] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:52.679926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:52.681028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:52.681425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:52.681866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] PATH_INFO: `/v3/projects/8608c1c9151d489997547a678e73abbf` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:52.708969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8608c1c9151d489997547a678e73abbf)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:52.715487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:52.768803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:52.770192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:52.851367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8608c1c9151d489997547a678e73abbf', 'request_id': 'req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:52.865527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8608c1c9151d489997547a678e73abbf', 'request_id': 'req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:52.878643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8608c1c9151d489997547a678e73abbf', 'request_id': 'req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:52.896907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:52.923172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] The token cache is being invalidate because project 8608c1c9151d489997547a678e73abbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:52.926262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:52.928035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:52.934020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1161/2294] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:52 2022] DELETE /identity/v3/projects/8608c1c9151d489997547a678e73abbf => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:52.938667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:52.940951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:52.971650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023964da-4bf6-4d63-8859-be70ff783591 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:53.106070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae19c-0e21-4b04-82d5-c5d2875df38a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:53.115478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:53.117431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:53.117891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:53.118348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] PATH_INFO: `/v3/users/8c0aa23266a04320bedc88e63d57777e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:53.120470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0aa23266a04320bedc88e63d57777e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:53.124170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:53.126973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:53.129393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:53.130717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:53.131175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:53.138942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:53.218681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0aa23266a04320bedc88e63d57777e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:53.244088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0aa23266a04320bedc88e63d57777e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:53.244971 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:53.246335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0aa23266a04320bedc88e63d57777e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:53.285164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0aa23266a04320bedc88e63d57777e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:53.295079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:53.297455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:53.306986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-40f7169a-90f4-431c-9a53-0e1615933d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0aa23266a04320bedc88e63d57777e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:53.322972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2296] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:32:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 12:32:53.364236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1162/2296] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:52 2022] DELETE /identity/v3/users/8c0aa23266a04320bedc88e63d57777e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:54.119729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:54.137053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adeaa2-ee43-4423-ac71-e22fec5d6e75 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:54.365488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:54.366920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:54.368430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:54.370124 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] PATH_INFO: `/v3/projects/4f59a894cc964acaac7fabe8821cdc46` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:54.372499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f59a894cc964acaac7fabe8821cdc46)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:54.379471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:54.523644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f59a894cc964acaac7fabe8821cdc46', 'request_id': 'req-d5b85a33-8f85-40cd-9de3-bc197a1e802f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:54.530688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f59a894cc964acaac7fabe8821cdc46', 'request_id': 'req-d5b85a33-8f85-40cd-9de3-bc197a1e802f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:54.549337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f59a894cc964acaac7fabe8821cdc46', 'request_id': 'req-d5b85a33-8f85-40cd-9de3-bc197a1e802f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:54.570069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] The token cache is being invalidate because project 4f59a894cc964acaac7fabe8821cdc46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:54.570627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:54.572879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d5b85a33-8f85-40cd-9de3-bc197a1e802f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:54.574794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2297] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:54 2022] DELETE /identity/v3/projects/4f59a894cc964acaac7fabe8821cdc46 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:54.581257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efe049-3c47-45d4-b55f-b662f4c4c9ba None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:54.582209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efe049-3c47-45d4-b55f-b662f4c4c9ba None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:54.582662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efe049-3c47-45d4-b55f-b662f4c4c9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:54.621752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-45efe049-3c47-45d4-b55f-b662f4c4c9ba None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:54.626422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-45efe049-3c47-45d4-b55f-b662f4c4c9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:54.744473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:54.745615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:54.746050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:54.768070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:54.770154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:54.778565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1163/2298] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:54.781136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:54.781533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:54.781916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:54.852435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:54.856289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:54.879878 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1164/2299] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:54.897292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:54.937519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce26a9-8a47-4a3b-9c95-d6c95404c28f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:54.949153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:54.950007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:54.950297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:54.950579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:54.952100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:32:54.952446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:54.954368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:54.974429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:54.984303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1165/2300] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:32:54.985625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:54.992932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2301] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jul 19 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:54.995159 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:54.999987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:55.007862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11057b2-1281-4d72-aa4e-3907b233e6a5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.013995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.018222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.018730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.019268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.020321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.022953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.053614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:55.104273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1166/2302] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:54 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:55.108915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.180856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7135b71-e4b2-4209-853f-3d344ffc9112 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.224487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01891323-c3e3-41a6-b73d-405bd483e84b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.232726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.233642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.234354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.234965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.235807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.239581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.272250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1167/2303] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:55 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:55.276587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.306412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb0e3b-8730-42a5-8067-239663269d9e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.309068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63585e7c-3ac2-4796-811c-2b1fef51280e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.309804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585e7c-3ac2-4796-811c-2b1fef51280e service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.310154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585e7c-3ac2-4796-811c-2b1fef51280e service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.310487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585e7c-3ac2-4796-811c-2b1fef51280e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.314896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.315773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.316432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.316908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.317831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.320833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.334845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1168/2304] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:55.340880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.355253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033a059-2209-4c3a-9b10-ebe2723d95f7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.362558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.363359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.363704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.364055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.364919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.367249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.392907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1169/2305] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:55.393819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63585e7c-3ac2-4796-811c-2b1fef51280e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.393965 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.397087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63585e7c-3ac2-4796-811c-2b1fef51280e service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.407287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb8be2-f96c-4584-91d2-2400d591e26c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.410211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2306] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:32:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 12:32:55.424398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.426213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.426501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.426787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] PATH_INFO: `/v3/projects/dce7f74dcf31413c9fdb0c2b21298831/users/c2a90b29f6f940bf9eea5aed5e186e4b/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.447499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce7f74dcf31413c9fdb0c2b21298831, user_id=c2a90b29f6f940bf9eea5aed5e186e4b, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.451147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.475156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1170/2307] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:55 2022] PUT /identity/v3/projects/dce7f74dcf31413c9fdb0c2b21298831/users/c2a90b29f6f940bf9eea5aed5e186e4b/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:55.486747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cdc77-8a3a-400c-b990-2eab5ee57de0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.487040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cdc77-8a3a-400c-b990-2eab5ee57de0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.487323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cdc77-8a3a-400c-b990-2eab5ee57de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.526177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c88cdc77-8a3a-400c-b990-2eab5ee57de0 None None] MFA Rules not processed for user `c2a90b29f6f940bf9eea5aed5e186e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:55.528655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c88cdc77-8a3a-400c-b990-2eab5ee57de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:55.603761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:55.636712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2308] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:55.640991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.734585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a567c7-b292-4971-8c2b-a95e91bab235 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.743176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.743176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.743602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.743602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.744847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.747977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.771064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.771438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.771744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.794423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1171/2309] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:32:55 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:55.798148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.805067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:55.806530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:55.812073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:55.906915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2310] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:32:55.922571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.925383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d541870-c085-430f-8150-9e891608a9be admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:55.933305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:55.934764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:55.935147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:55.935434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:55.936966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:55.940307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:55.973896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1172/2311] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:32:55 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:32:55.975075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:55.992969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f899-fad6-49ce-8590-5525cf177215 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:56.003563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:56.004640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.005373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.005691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.006660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:56.008818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:56.023289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1173/2312] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:32:56.025185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:56.036274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7575a-d05f-42b5-87db-96919476d2c6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:56.042828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:56.043325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.043624 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.043910 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] PATH_INFO: `/v3/projects/91fb17c9bd974324a3849d8831acb305/users/2c8b04ba4b334419b6e758de35b0d528/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.065966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fb17c9bd974324a3849d8831acb305, user_id=2c8b04ba4b334419b6e758de35b0d528, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:56.069421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:56.083754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f5638-fa18-4f1a-b453-1fcada49ee34 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:56.093575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1174/2313] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:56 2022] PUT /identity/v3/projects/91fb17c9bd974324a3849d8831acb305/users/2c8b04ba4b334419b6e758de35b0d528/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:56.097715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:56.190870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848d836-2318-49ca-9ed5-db510bbfcf89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:56.191702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848d836-2318-49ca-9ed5-db510bbfcf89 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.191993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848d836-2318-49ca-9ed5-db510bbfcf89 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.192312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848d836-2318-49ca-9ed5-db510bbfcf89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.217839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c974-62dd-41b1-9011-23246754201e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:56.223126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:56.223810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.224098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.224566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.225224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:56.231912 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:56.258406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:56.265131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1175/2314] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:32:56.266769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:56.291291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b74f1-900d-4814-b436-08814124b3f6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:56.317250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848d836-2318-49ca-9ed5-db510bbfcf89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:56.318025 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:56.319948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.320270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.321551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848d836-2318-49ca-9ed5-db510bbfcf89 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:56.328269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] PATH_INFO: `/v3/projects/91fb17c9bd974324a3849d8831acb305/users/2c8b04ba4b334419b6e758de35b0d528/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.334257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fb17c9bd974324a3849d8831acb305, user_id=2c8b04ba4b334419b6e758de35b0d528, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:56.337048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:56.338233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2315] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:32:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 12:32:56.372472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1176/2316] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:32:56 2022] PUT /identity/v3/projects/91fb17c9bd974324a3849d8831acb305/users/2c8b04ba4b334419b6e758de35b0d528/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:56.383540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f83d8-bebd-4e62-a7d8-7e1e572d5f9a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.384064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f83d8-bebd-4e62-a7d8-7e1e572d5f9a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.384489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f83d8-bebd-4e62-a7d8-7e1e572d5f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.429247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-384f83d8-bebd-4e62-a7d8-7e1e572d5f9a None None] MFA Rules not processed for user `2c8b04ba4b334419b6e758de35b0d528`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:56.430522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-384f83d8-bebd-4e62-a7d8-7e1e572d5f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:56.621820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2317] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:56.633451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:56.635199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:56.719007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:56.752439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87c14-b790-4105-a144-d7ddd1eba550 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:56.757682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:56.758461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.758847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.759225 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.778413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:56.780771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:56.803720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1177/2318] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:56.897134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:56.897797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:56.898254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:56.926459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:56.927822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:56.977784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:57.001638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2319] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jul 19 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:57.014832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:57.052109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d7b0e-5588-4c0d-ae17-2dd6c62c857a service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:57.058661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:57.059667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:57.060058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:57.060475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] PATH_INFO: `/v3/users/c2a90b29f6f940bf9eea5aed5e186e4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:57.061834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2a90b29f6f940bf9eea5aed5e186e4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:57.064133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:57.090842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90597db-cd1d-4136-a4cc-6c6824d9bd96 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:57.096702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac339fe-9b69-4763-8482-f6a011cc111f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:57.097441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac339fe-9b69-4763-8482-f6a011cc111f service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:57.097863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac339fe-9b69-4763-8482-f6a011cc111f service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:57.098217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac339fe-9b69-4763-8482-f6a011cc111f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:57.108531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac339fe-9b69-4763-8482-f6a011cc111f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:57.110232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac339fe-9b69-4763-8482-f6a011cc111f service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:57.114712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:57.120877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2320] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:32:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 19 12:32:57.126064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2a90b29f6f940bf9eea5aed5e186e4b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:57.137595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2a90b29f6f940bf9eea5aed5e186e4b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:57.138145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2a90b29f6f940bf9eea5aed5e186e4b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:57.158786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2a90b29f6f940bf9eea5aed5e186e4b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:57.167203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2a90b29f6f940bf9eea5aed5e186e4b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:57.173871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1178/2321] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:56 2022] DELETE /identity/v3/users/c2a90b29f6f940bf9eea5aed5e186e4b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:32:57.178537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:57.178827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:57.179113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:57.197615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:32:57.198925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:57.274487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2322] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:32:57.285945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:57.287769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:57.371071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4258d87-325a-42b6-9256-0bfb3c164848 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:57.376346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:57.377061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:57.377361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:57.380741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:57.407570 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:57.409963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:57.426559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1179/2323] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:32:59.212804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:59.214388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:59.239336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3bb0-7ecc-42b7-a37b-b2469f5aab88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:59.249235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:59.251181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:59.251861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:59.252383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] PATH_INFO: `/v3/projects/dce7f74dcf31413c9fdb0c2b21298831` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:59.254525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce7f74dcf31413c9fdb0c2b21298831)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:59.257108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:59.363388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce7f74dcf31413c9fdb0c2b21298831', 'request_id': 'req-fbf7e443-899d-4885-abbf-df15aa5b2734'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.381104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce7f74dcf31413c9fdb0c2b21298831', 'request_id': 'req-fbf7e443-899d-4885-abbf-df15aa5b2734'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.409609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce7f74dcf31413c9fdb0c2b21298831', 'request_id': 'req-fbf7e443-899d-4885-abbf-df15aa5b2734'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.426383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] The token cache is being invalidate because project dce7f74dcf31413c9fdb0c2b21298831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:32:59.426680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.427479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.432105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2324] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:32:59 2022] DELETE /identity/v3/projects/dce7f74dcf31413c9fdb0c2b21298831 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:59.433532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:59.456026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7bddd-387a-476e-ad80-d22e329f55c0 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:59.475273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:32:59.602156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:59.604094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:59.604445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:59.604760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] PATH_INFO: `/v3/users/2c8b04ba4b334419b6e758de35b0d528` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:59.606062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8b04ba4b334419b6e758de35b0d528)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:59.608309 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:32:59.718676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8b04ba4b334419b6e758de35b0d528'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.730432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8b04ba4b334419b6e758de35b0d528'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.731400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8b04ba4b334419b6e758de35b0d528'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.755331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8b04ba4b334419b6e758de35b0d528'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.764645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c5f1635f-a4ea-458a-a0da-51ba41ba5ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8b04ba4b334419b6e758de35b0d528'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:32:59.772513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1180/2325] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:59 2022] DELETE /identity/v3/users/2c8b04ba4b334419b6e758de35b0d528 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:32:59.877098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:32:59.878820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:32:59.896927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7e443-899d-4885-abbf-df15aa5b2734 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:32:59.985544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:32:59.986099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:32:59.986583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:32:59.986935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] PATH_INFO: `/v3/users/c9440928fa144c25a9f959e28cd6e866` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:32:59.989979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9440928fa144c25a9f959e28cd6e866)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:32:59.991960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:00.019644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:00.051144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9440928fa144c25a9f959e28cd6e866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.051960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9440928fa144c25a9f959e28cd6e866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.088725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9440928fa144c25a9f959e28cd6e866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.096727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9440928fa144c25a9f959e28cd6e866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.126219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9440928fa144c25a9f959e28cd6e866'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.138252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2326] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:32:59 2022] DELETE /identity/v3/users/c9440928fa144c25a9f959e28cd6e866 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:33:00.141327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:00.141994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:00.142740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:00.175095 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:00.176648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:00.278666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1181/2327] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:00.288975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:00.290736 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:00.401734 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d559c-8bef-4238-ba32-a24f4d7e42ac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:00.407754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:00.408609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:00.408992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:00.409379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:00.422641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:00.424415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:00.436258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2328] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:00.714908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:00.718620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:00.772338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c513e-bc68-42fe-a01b-359d29737ca2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:00.782285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:00.783609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:00.783892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:00.784168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] PATH_INFO: `/v3/projects/e9181ed0a5c14a54b7d15417e147ee11` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:00.787751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9181ed0a5c14a54b7d15417e147ee11)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:00.792592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:00.792772 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:00.798346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:00.829252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217022f-6188-45b5-8d1a-544754df83de service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:00.834160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:00.835156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:00.835541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:00.835934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] PATH_INFO: `/v3/projects/91fb17c9bd974324a3849d8831acb305` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:00.837444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=91fb17c9bd974324a3849d8831acb305)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:00.839786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:00.861587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:00.898255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9181ed0a5c14a54b7d15417e147ee11', 'request_id': 'req-a8286b4c-153a-4765-9b39-5d25ed3f6256'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.912625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9181ed0a5c14a54b7d15417e147ee11', 'request_id': 'req-a8286b4c-153a-4765-9b39-5d25ed3f6256'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.921898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9181ed0a5c14a54b7d15417e147ee11', 'request_id': 'req-a8286b4c-153a-4765-9b39-5d25ed3f6256'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.931117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91fb17c9bd974324a3849d8831acb305', 'request_id': 'req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.934189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] The token cache is being invalidate because project e9181ed0a5c14a54b7d15417e147ee11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:00.935861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.939117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a8286b4c-153a-4765-9b39-5d25ed3f6256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.943003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1182/2329] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:00 2022] DELETE /identity/v3/projects/e9181ed0a5c14a54b7d15417e147ee11 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:00.948280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91fb17c9bd974324a3849d8831acb305', 'request_id': 'req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.948949 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c253-7b58-4127-a5c5-18586c2536ee None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:00.949527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c253-7b58-4127-a5c5-18586c2536ee None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:00.950262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c253-7b58-4127-a5c5-18586c2536ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:00.962920 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91fb17c9bd974324a3849d8831acb305', 'request_id': 'req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.974788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a339c253-7b58-4127-a5c5-18586c2536ee None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:00.976019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a339c253-7b58-4127-a5c5-18586c2536ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:00.994582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] The token cache is being invalidate because project 91fb17c9bd974324a3849d8831acb305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:00.995382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.996719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ec2fe631-5683-4d16-a0f2-f36eb1c830fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:00.999907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2330] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:00 2022] DELETE /identity/v3/projects/91fb17c9bd974324a3849d8831acb305 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:01.005033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ea4f-d078-4033-8ef8-5f19d6e948d5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.005536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ea4f-d078-4033-8ef8-5f19d6e948d5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.006096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ea4f-d078-4033-8ef8-5f19d6e948d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.038906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30e6ea4f-d078-4033-8ef8-5f19d6e948d5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:01.041384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30e6ea4f-d078-4033-8ef8-5f19d6e948d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.073811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:01.099897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1183/2331] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:33:01.104945 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.105250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.105533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.140711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:01.142987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.157322 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1184/2332] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:01.161250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.161822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2333] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:33:01.170205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.170495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.170770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.173103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d621bb3-9f7c-4570-a04d-d4a8c5066450 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.179927 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.180635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.180919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.181198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.181933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:01.182240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.183883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.191601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:01.193267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.200811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1185/2334] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:33:01.204877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.206195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2335] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:01.210177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.215840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb661fb3-9b89-49bd-802d-93adfd75aaea admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.223333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.223533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c92e83-e56c-4e57-9014-fe3dba1945ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.224046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.224505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.224785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.225481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.230501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.231349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.231787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.232229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.233393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:01.233970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.237190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.242613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.260041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2336] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:33:01.262863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.265677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.280675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf73583-8b0f-4985-ae90-fcc318c01cd6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.285701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.286873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.287243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.287593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] PATH_INFO: `/v3/users/1f34d29c3fa647d89dfce583e1d40705` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.289510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f34d29c3fa647d89dfce583e1d40705)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.291635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.304825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1186/2337] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:01.306494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.361460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f34d29c3fa647d89dfce583e1d40705'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:01.362637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f34d29c3fa647d89dfce583e1d40705'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:01.379749 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f34d29c3fa647d89dfce583e1d40705'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:01.394600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f34d29c3fa647d89dfce583e1d40705'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:01.423527 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f34d29c3fa647d89dfce583e1d40705'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:01.430746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe30e8-7d8d-4e5d-9e25-e17d7bcffe12 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.438398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2338] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:01 2022] DELETE /identity/v3/users/1f34d29c3fa647d89dfce583e1d40705 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:01.440727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.444540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.444892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7d9e3-e852-4938-b93d-4948c366fcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.445793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7d9e3-e852-4938-b93d-4948c366fcdd admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.447342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7d9e3-e852-4938-b93d-4948c366fcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.447797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7d9e3-e852-4938-b93d-4948c366fcdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.449637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7d9e3-e852-4938-b93d-4948c366fcdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.453244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7d9e3-e852-4938-b93d-4948c366fcdd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.481607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:01.500478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1187/2339] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/projects => generated 407 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:01.502118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.502578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.502992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.523346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:01.524908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.565672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9053-add9-4f9f-a693-a5a1592ef91c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.597361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:01.638258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1188/2340] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:01.639911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.641536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.659302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1afe7-2f5d-4138-8328-e78e1092e903 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.666179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.669635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.671395 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.672443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.675150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.681851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.683259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.683568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.683912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.689110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.692430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.695535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.725145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2341] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/users => generated 300 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:01.726053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.730025 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:01.736312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1189/2342] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:01 2022] POST /identity/v3/users => generated 304 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:01.737900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.751165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729506a8-62b3-4d47-9649-6775fba0124a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.757507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.758228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.758611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.758960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.759776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.761617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.773768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1190/2343] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:01.774509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.788291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad12ed8-16b1-46c2-9b64-c47ff3b77fea admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.794953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.796650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.797150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.797641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.799084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.801982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.816781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1191/2344] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:01.817731 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.835771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a219094-b77c-49fc-9cd2-96b3edfe3aae admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.837704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d3cd-abc1-4fc1-bb6c-919e8cb40a2e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.844883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.846181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.846455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.847746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.849246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.850328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.851834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.852427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.852721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.853147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.868675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1192/2345] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:01.868675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.877214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.881635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a82dd1-5d74-4783-a3a8-bd5a7dc40d7a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.889218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.890063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.890355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.890831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.890934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.891505 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.893515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.908872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2346] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:01.910769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.929553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1193/2347] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:01.930461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00540b8-7c90-4e55-8dfd-af58327d13f8 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.935410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:01.938425 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e1cd9-b859-4824-a300-367b12e6b207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.939510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e1cd9-b859-4824-a300-367b12e6b207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.939940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e1cd9-b859-4824-a300-367b12e6b207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.940462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e1cd9-b859-4824-a300-367b12e6b207 admin admin] PATH_INFO: `/v3/projects/537f77ec01ed474c88c47eb4b758e515/users/0fd0f22cd24546b0a8f7eb519e1969df/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:01.970204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2ad07-1a98-4685-8123-1e6addbf067d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:01.973894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e1cd9-b859-4824-a300-367b12e6b207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=537f77ec01ed474c88c47eb4b758e515, user_id=0fd0f22cd24546b0a8f7eb519e1969df, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:01.979181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e1cd9-b859-4824-a300-367b12e6b207 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:01.983012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c15b1-973d-406b-945e-dd66b662c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:01.984431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c15b1-973d-406b-945e-dd66b662c3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:01.984764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c15b1-973d-406b-945e-dd66b662c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:01.986529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c15b1-973d-406b-945e-dd66b662c3e7 admin admin] PATH_INFO: `/v3/projects/ba0bd89894fa4cc0853ab49507af7d9a/users/2d6df37575fd47acb7f055ea446859c4/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.015755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2348] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:01 2022] PUT /identity/v3/projects/537f77ec01ed474c88c47eb4b758e515/users/0fd0f22cd24546b0a8f7eb519e1969df/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:02.015755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.015755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.015755 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.016057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c15b1-973d-406b-945e-dd66b662c3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0bd89894fa4cc0853ab49507af7d9a, user_id=2d6df37575fd47acb7f055ea446859c4, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.035652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c15b1-973d-406b-945e-dd66b662c3e7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:02.056778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] MFA Rules not processed for user `0fd0f22cd24546b0a8f7eb519e1969df`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:02.059125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:02.068938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1194/2349] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:01 2022] PUT /identity/v3/projects/ba0bd89894fa4cc0853ab49507af7d9a/users/2d6df37575fd47acb7f055ea446859c4/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:02.081050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.081435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.081717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.134442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] MFA Rules not processed for user `2d6df37575fd47acb7f055ea446859c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:02.137289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:02.261865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2350] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:02.274983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:02.276157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:02.281378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:02.291880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1195/2351] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:02.297335 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:02.335586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:02.412386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e698332-6ed5-435a-9641-80879c67b397 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:02.418973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:02.419828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.420785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.422480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.458659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2298329-82c6-4903-b35b-2bfd250ffc9e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:02.462630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.469147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:02.472652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:02.473464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.473857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.474239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.475136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.477443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:02.497134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2352] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:02.558412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:02.559401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1196/2353] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:02 2022] POST /identity/v3/projects => generated 407 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:02.579628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fec0-e582-4d75-a088-6dcea56fa8e1 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:02.730812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:02.731568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.731784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.732070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.734218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.736205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:02.788996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2354] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:02 2022] POST /identity/v3/users => generated 310 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:02.793790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:02.799307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:02.802877 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:02.811118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824f154-7a21-4a5e-a8f6-9ccab1bb45e3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:02.823474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61c165-9919-4e08-ace6-61c03163c962 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:02.834343 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:02.835688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.835975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.836647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.838444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:02.838587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.840616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:02.847523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.847831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.858359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] PATH_INFO: `/v3/projects/c3ff2fe9ee5d4784bdb104a2e750f8e6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.862916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ff2fe9ee5d4784bdb104a2e750f8e6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.871389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1197/2355] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:02.876087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:02.878275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:02.914238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6245286-bc2d-4a12-9b8d-92bff8a268c4 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:02.929636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:02.931457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:02.933131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:02.933593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] PATH_INFO: `/v3/projects/c5b09dda5f624bd884bf2e50c276ddff/users/64e5b41a75354c77a9525faf5f59ce21/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:02.968972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b09dda5f624bd884bf2e50c276ddff, user_id=64e5b41a75354c77a9525faf5f59ce21, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:02.973719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:03.005818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1198/2356] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:02 2022] PUT /identity/v3/projects/c5b09dda5f624bd884bf2e50c276ddff/users/64e5b41a75354c77a9525faf5f59ce21/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:03.010979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.042504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ff2fe9ee5d4784bdb104a2e750f8e6', 'request_id': 'req-ab9e8efb-f456-4ce4-848c-8b651883c49d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:03.058029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ff2fe9ee5d4784bdb104a2e750f8e6', 'request_id': 'req-ab9e8efb-f456-4ce4-848c-8b651883c49d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:03.075051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ff2fe9ee5d4784bdb104a2e750f8e6', 'request_id': 'req-ab9e8efb-f456-4ce4-848c-8b651883c49d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:03.106897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] The token cache is being invalidate because project c3ff2fe9ee5d4784bdb104a2e750f8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:03.107225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:03.108473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ab9e8efb-f456-4ce4-848c-8b651883c49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:03.111316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2357] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:02 2022] DELETE /identity/v3/projects/c3ff2fe9ee5d4784bdb104a2e750f8e6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:03.117940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83827c19-32e5-4ec4-98a3-36576302afb6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.118307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83827c19-32e5-4ec4-98a3-36576302afb6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.118622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83827c19-32e5-4ec4-98a3-36576302afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.145694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-83827c19-32e5-4ec4-98a3-36576302afb6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:03.147721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83827c19-32e5-4ec4-98a3-36576302afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:03.205205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e021-f4ff-4372-adea-3a17a17143d7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:03.219886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:03.269599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2358] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:03.278258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.278931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.279312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.287315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:03.300308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:03.301034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.301327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.301608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.302350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:03.303987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:03.333967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:03.335442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:03.344226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1199/2359] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:03.345331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.357444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:03.367168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2360] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:03.371848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.393415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747080-932b-43de-b34d-62e6dac67b2d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:03.400162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede8722-f1c7-40ba-a883-2900532f6b92 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:03.402989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:03.403931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.405153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.405561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.406374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:03.406791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:03.409621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:03.414103 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48c257-b7ae-459f-b944-b2109914e4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:03.415024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48c257-b7ae-459f-b944-b2109914e4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.415314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48c257-b7ae-459f-b944-b2109914e4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.416308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48c257-b7ae-459f-b944-b2109914e4cb admin admin] PATH_INFO: `/v3/projects/c5b09dda5f624bd884bf2e50c276ddff/users/64e5b41a75354c77a9525faf5f59ce21/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.425597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48c257-b7ae-459f-b944-b2109914e4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b09dda5f624bd884bf2e50c276ddff, user_id=64e5b41a75354c77a9525faf5f59ce21, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:03.429514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48c257-b7ae-459f-b944-b2109914e4cb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:03.432157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2361] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:33:03.440139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.458769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1200/2362] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:03 2022] PUT /identity/v3/projects/c5b09dda5f624bd884bf2e50c276ddff/users/64e5b41a75354c77a9525faf5f59ce21/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:03.471271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.472065 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.472740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.473137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5722-f9aa-4359-ae55-b85dcc532143 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:03.541353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] MFA Rules not processed for user `64e5b41a75354c77a9525faf5f59ce21`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:03.542680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:03.558135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2dbf81-7eb4-4ca8-be78-8beb2a3a2e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:03.559782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dbf81-7eb4-4ca8-be78-8beb2a3a2e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.560363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dbf81-7eb4-4ca8-be78-8beb2a3a2e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.562643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2dbf81-7eb4-4ca8-be78-8beb2a3a2e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.566897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dbf81-7eb4-4ca8-be78-8beb2a3a2e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:03.570250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2dbf81-7eb4-4ca8-be78-8beb2a3a2e2d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:03.653977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2363] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:03.655631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.655967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.656238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.692814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] MFA Rules not processed for user `880c0417a58d48ceaab3912957f2e4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:03.694194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:03.751801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1201/2364] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:03.752286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.776680 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b273-fdd0-4819-ae37-1f816ef0fee2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:03.831232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2365] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:03.831232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.839323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:03.878543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:03.879238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:03.879534 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:03.879806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:03.880535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:03.887830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:03.921925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:03.932490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1202/2366] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:03.933454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:03.937017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:04.000576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22603bcc-cff7-4ae0-81c4-e81d7fc22f50 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.008914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.009749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.010039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.010328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.011038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.012998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.025103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e219-213b-492d-90e5-27bbefe286c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.043910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.047400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.047832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.048385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.074386 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1203/2367] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:03 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:04.075668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.158314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:04.230451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.234074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.238895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6736ec0-65cb-4e62-8112-88092e05ef92 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.243908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.244630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.244917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.245200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.245940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.247716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.260233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2368] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:04.261013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1204/2369] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:04.261168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.277023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.290136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea362f6d-7a63-4695-86e6-2d562c5d38f2 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.301475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1199d-67cd-48aa-aded-a0f83f07d0e7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.305024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.305718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.306006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.307331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.308737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.309613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.309999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.310396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.311277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.312152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.314657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.322604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.331334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1205/2370] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:04.334856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.353442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e608ecb-ae10-4dd9-93be-a88fa11f6437 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.361371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.368056 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.368284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.371082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] PATH_INFO: `/v3/projects/6a2288a8993743e8a11d244b25bd1fcd/users/61f8d4d414af4a3b8f7050cb012d6454/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.375579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2371] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:04 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:04.379141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.402203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea06e9-1559-497e-a360-9c3632b79f05 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.412149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2288a8993743e8a11d244b25bd1fcd, user_id=61f8d4d414af4a3b8f7050cb012d6454, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.413745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebc31d-1556-492c-8063-af5f540c0f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.416130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebc31d-1556-492c-8063-af5f540c0f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.417663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebc31d-1556-492c-8063-af5f540c0f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.418098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebc31d-1556-492c-8063-af5f540c0f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.419844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebc31d-1556-492c-8063-af5f540c0f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.420027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.424438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebc31d-1556-492c-8063-af5f540c0f4e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.445124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1206/2372] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:04 2022] PUT /identity/v3/projects/6a2288a8993743e8a11d244b25bd1fcd/users/61f8d4d414af4a3b8f7050cb012d6454/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:04.449468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2373] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:04.462932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.473145 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.474500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.475537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.580685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] MFA Rules not processed for user `61f8d4d414af4a3b8f7050cb012d6454`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:04.584440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:04.652872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070d81d-1837-4280-8f89-32e39954e8e5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.666493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.668502 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.670660 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.673193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] PATH_INFO: `/v3/projects/a0b1f6fd65a24bdf84678d6fa552ba22/users/591611d1191e4e5abe9008003cae285a/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.707667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b1f6fd65a24bdf84678d6fa552ba22, user_id=591611d1191e4e5abe9008003cae285a, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:04.713471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:04.747052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1207/2374] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:04 2022] PUT /identity/v3/projects/a0b1f6fd65a24bdf84678d6fa552ba22/users/591611d1191e4e5abe9008003cae285a/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:04.752473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.757240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:04.757403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2375] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:04.757403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:04.786889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b47a7-56ac-4ad5-a823-73c26ea64241 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.900913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767193d-a828-4164-8878-2f27c55fb710 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:04.914124 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:04.917798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:04.918101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:04.930792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:04.931872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:04.941288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:05.014314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.015014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.015303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.015587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.016353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.018014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.064403 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2376] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:05.065387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:05.080710 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:05.095386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c0bce-1563-4dc6-8436-cad67befe8b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:05.107906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.108946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.109245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.109530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.113989 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.116853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.159908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2377] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:05.160832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:05.176144 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.177673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.200825 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1208/2378] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:05.210972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:05.249232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7e975-01a9-407a-b595-68d7f2b0288e service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:05.297492 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fec490-be25-4853-9c04-2a77d4d34dd2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:05.317680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.318628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.318901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.319186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] PATH_INFO: `/v3/projects/a0b1f6fd65a24bdf84678d6fa552ba22/users/591611d1191e4e5abe9008003cae285a/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.330205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b1f6fd65a24bdf84678d6fa552ba22, user_id=591611d1191e4e5abe9008003cae285a, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.338699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.375356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2379] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:05 2022] PUT /identity/v3/projects/a0b1f6fd65a24bdf84678d6fa552ba22/users/591611d1191e4e5abe9008003cae285a/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:05.426172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.429237 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.431113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.431826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.433377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.440859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.486149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:05.495567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1209/2380] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:05 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:05.669004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021274-9ecd-49be-8c8d-928623434740 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:05.677001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.679306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.679825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.680213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.682341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.684336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.718043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:05.731123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2381] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:33:05.740315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35af37-e203-4111-88d5-b0e909f7b9d2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:05.746157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63d129-09cf-4a4d-91fb-889ce6962a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.747051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d129-09cf-4a4d-91fb-889ce6962a4f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.747410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d129-09cf-4a4d-91fb-889ce6962a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.747789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d129-09cf-4a4d-91fb-889ce6962a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.749176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63d129-09cf-4a4d-91fb-889ce6962a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.751149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63d129-09cf-4a4d-91fb-889ce6962a4f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.791135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:05.794274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1210/2382] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:05.808881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29839b-f761-4ede-b4e1-0b0e1edad33d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:05.823735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:05.824674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.825057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.825234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] PATH_INFO: `/v3/projects/20a3c7aa55f64f5d88d8d143f080a2d9/users/1c7bd6cd08a84a70a52249cb19a0e240/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:05.860295 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a3c7aa55f64f5d88d8d143f080a2d9, user_id=1c7bd6cd08a84a70a52249cb19a0e240, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:05.870577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:05.897100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2383] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:05 2022] PUT /identity/v3/projects/20a3c7aa55f64f5d88d8d143f080a2d9/users/1c7bd6cd08a84a70a52249cb19a0e240/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:33:05.911278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:05.911278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:05.914342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.008572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] MFA Rules not processed for user `1c7bd6cd08a84a70a52249cb19a0e240`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:06.009851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:06.140783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1211/2384] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:06.141462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:06.205864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:06.260065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4753e-adb4-45c5-a22a-f6f294c4e616 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:06.267607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:06.269021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.269537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.271744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.272868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:06.276786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:06.320112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2385] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:06 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:06.325297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:06.339846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897676c5-4da4-480d-b6dc-874ca72de531 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:06.416247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:06.490726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:06.492601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.496209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.496715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.498702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:06.502532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:06.540431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1212/2386] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:06 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:06.544169 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:06.561536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71936611-527e-4844-87d3-2bebaf359f79 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:06.567936 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:06.568707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.569060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.569344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.570147 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:06.573582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:06.596100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2387] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:06.613300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:06.633518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765cb67-68ca-44bb-91b8-412ab592ffee admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:06.642876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:06.644576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.645645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.646506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] PATH_INFO: `/v3/projects/c979f665603d407eb1d7362ac41d193a/users/c337ca2d26ee45af93f06ffcccd44176/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.665983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c979f665603d407eb1d7362ac41d193a, user_id=c337ca2d26ee45af93f06ffcccd44176, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:06.669697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:06.689093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1213/2388] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:06 2022] PUT /identity/v3/projects/c979f665603d407eb1d7362ac41d193a/users/c337ca2d26ee45af93f06ffcccd44176/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:06.693036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:06.705229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdbe7-3a6f-4061-870b-0b3124fc549c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:06.835600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd0c81-1d40-4319-989e-e90125aeb724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:06.836583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd0c81-1d40-4319-989e-e90125aeb724 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.837250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd0c81-1d40-4319-989e-e90125aeb724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.838127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd0c81-1d40-4319-989e-e90125aeb724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.839146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd0c81-1d40-4319-989e-e90125aeb724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:06.841643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd0c81-1d40-4319-989e-e90125aeb724 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:06.853559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2389] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:06.859051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:06.879215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516cc09-d517-4509-bf2e-bd7380cb1a2d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:06.893973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:06.895156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.895606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.896043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] PATH_INFO: `/v3/projects/c979f665603d407eb1d7362ac41d193a/users/c337ca2d26ee45af93f06ffcccd44176/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:06.900986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c979f665603d407eb1d7362ac41d193a, user_id=c337ca2d26ee45af93f06ffcccd44176, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:06.905979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:06.942966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1214/2390] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:06 2022] PUT /identity/v3/projects/c979f665603d407eb1d7362ac41d193a/users/c337ca2d26ee45af93f06ffcccd44176/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:06.954134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:06.954397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:06.955657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:07.005362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] MFA Rules not processed for user `c337ca2d26ee45af93f06ffcccd44176`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:07.007076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:07.130385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:07.131959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2391] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:07.225539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f6123-6da0-42f5-8f32-9d5e0b223cfd admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:07.241233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:07.241870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:07.242157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:07.242431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:07.339719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:07.341803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:07.358607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1215/2392] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:09.109754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:09.120339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde92658-0c98-4307-8a96-7f803b4744d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:09.125817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74926f-9612-4920-8541-04074ca9df00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:09.126478 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74926f-9612-4920-8541-04074ca9df00 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:09.127283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74926f-9612-4920-8541-04074ca9df00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:09.127578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74926f-9612-4920-8541-04074ca9df00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:09.153204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:09.229907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74926f-9612-4920-8541-04074ca9df00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:09.233982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74926f-9612-4920-8541-04074ca9df00 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:09.245927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2393] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:10.722935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:10.733323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaf275-8c0f-49c0-9eb0-4abfe8625fe2 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:10.753459 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:10.754297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:10.756475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:10.756846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:10.774511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:10.777994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:10.802349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1216/2394] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:11.550248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:11.550780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:11.550980 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:11.579437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] MFA Rules not processed for user `591611d1191e4e5abe9008003cae285a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:11.580745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:11.691084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2395] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:11.704585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:11.706074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:11.743623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:11.789840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592703cf-9f92-49da-b9e9-ca4fe11f1091 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:11.795919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bc4a7-e9df-4c93-9df8-28dc7863090f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:11.796610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bc4a7-e9df-4c93-9df8-28dc7863090f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:11.796894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bc4a7-e9df-4c93-9df8-28dc7863090f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:11.797173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bc4a7-e9df-4c93-9df8-28dc7863090f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:11.857477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bc4a7-e9df-4c93-9df8-28dc7863090f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:11.859107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bc4a7-e9df-4c93-9df8-28dc7863090f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:11.869568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1217/2396] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:12.731002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:12.733232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:12.884817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3eb86d-1d0d-4cf7-a9db-5b457cf6851b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:12.900079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:12.902895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:12.903583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:12.913097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] PATH_INFO: `/v3/users/0fd0f22cd24546b0a8f7eb519e1969df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:12.915023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd0f22cd24546b0a8f7eb519e1969df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:12.917498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:12.996757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd0f22cd24546b0a8f7eb519e1969df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:12.997605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd0f22cd24546b0a8f7eb519e1969df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:13.001443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:13.009793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd0f22cd24546b0a8f7eb519e1969df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:13.026418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd0f22cd24546b0a8f7eb519e1969df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:13.054542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd0f22cd24546b0a8f7eb519e1969df'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:13.064223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2397] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:12 2022] DELETE /identity/v3/users/0fd0f22cd24546b0a8f7eb519e1969df => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:13.069075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:13.069581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:13.069967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:13.117900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:13.123159 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:13.285810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1218/2398] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:13.295874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:13.297773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:13.384033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1522-6fbb-4e18-92d1-932c01cb4d60 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:13.388553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:13.389318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:13.389673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:13.390019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:13.406362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:13.408144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:13.451673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2399] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:14.731725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:14.734911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:14.817635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5883a1-75c9-4fce-9376-9f3acf63adf8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:14.834797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:14.835891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:14.837190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:14.837650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] PATH_INFO: `/v3/projects/537f77ec01ed474c88c47eb4b758e515` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:14.839183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=537f77ec01ed474c88c47eb4b758e515)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:14.842055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:14.928557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '537f77ec01ed474c88c47eb4b758e515', 'request_id': 'req-4be95b42-356c-4a83-89c2-3d9e470cca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:14.941777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '537f77ec01ed474c88c47eb4b758e515', 'request_id': 'req-4be95b42-356c-4a83-89c2-3d9e470cca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:14.953087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '537f77ec01ed474c88c47eb4b758e515', 'request_id': 'req-4be95b42-356c-4a83-89c2-3d9e470cca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:14.956205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:14.971108 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] The token cache is being invalidate because project 537f77ec01ed474c88c47eb4b758e515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:14.972076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:14.973163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4be95b42-356c-4a83-89c2-3d9e470cca85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:14.976250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1219/2400] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:14 2022] DELETE /identity/v3/projects/537f77ec01ed474c88c47eb4b758e515 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:15.729061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:15.730556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:15.747842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9a0f-9947-4143-9456-ecd0235a0d68 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:15.816141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:15.817162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:15.820499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:15.820804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] PATH_INFO: `/v3/users/2d6df37575fd47acb7f055ea446859c4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:15.823697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6df37575fd47acb7f055ea446859c4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:15.825680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:15.894188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6df37575fd47acb7f055ea446859c4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:15.894987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6df37575fd47acb7f055ea446859c4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:15.905343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6df37575fd47acb7f055ea446859c4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:15.916068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6df37575fd47acb7f055ea446859c4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:15.924260 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:15.933383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6df37575fd47acb7f055ea446859c4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:15.941038 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2401] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:15 2022] DELETE /identity/v3/users/2d6df37575fd47acb7f055ea446859c4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:15.946093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:15.946579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:15.946753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:15.970644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:15.971873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:16.038403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1220/2402] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:16.050821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:16.052539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:16.113044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51de89e-d3fe-431f-81d3-5ba62535531a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:16.116787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:16.117439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:16.117745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:16.118021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:16.132841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:16.134303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:16.145096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2403] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:16.493023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:16.494619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:16.531027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9308e-9d97-448d-82fb-8ec9f0d804a8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:16.541313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:16.542208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:16.542499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:16.542783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] PATH_INFO: `/v3/projects/ba0bd89894fa4cc0853ab49507af7d9a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:16.546097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0bd89894fa4cc0853ab49507af7d9a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:16.549346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:16.583115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:16.584722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:16.609579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a641583-45ac-4fe3-80f7-f97f2c9a0c22 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:16.626309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:16.627682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:16.628391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:16.628735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] PATH_INFO: `/v3/users/61f8d4d414af4a3b8f7050cb012d6454` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:16.632219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f8d4d414af4a3b8f7050cb012d6454)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:16.637660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:16.690740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0bd89894fa4cc0853ab49507af7d9a', 'request_id': 'req-1076af3b-b3d8-496f-878c-b50b01b84f24'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.704981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0bd89894fa4cc0853ab49507af7d9a', 'request_id': 'req-1076af3b-b3d8-496f-878c-b50b01b84f24'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.717521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0bd89894fa4cc0853ab49507af7d9a', 'request_id': 'req-1076af3b-b3d8-496f-878c-b50b01b84f24'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.735782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] The token cache is being invalidate because project ba0bd89894fa4cc0853ab49507af7d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:16.736327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.737289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.740304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1221/2404] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:16 2022] DELETE /identity/v3/projects/ba0bd89894fa4cc0853ab49507af7d9a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:16.744094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:16.750526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:16.764714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f8d4d414af4a3b8f7050cb012d6454'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.768071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f8d4d414af4a3b8f7050cb012d6454'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.797238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f8d4d414af4a3b8f7050cb012d6454'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.813286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f8d4d414af4a3b8f7050cb012d6454'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.852415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-888afab0-bf83-45c8-8c4b-3bb412aa4b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f8d4d414af4a3b8f7050cb012d6454'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:16.879642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2405] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:16 2022] DELETE /identity/v3/users/61f8d4d414af4a3b8f7050cb012d6454 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:33:16.881183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:16.881503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:16.881745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:16.913669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:16.914842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:16.930321 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076af3b-b3d8-496f-878c-b50b01b84f24 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:16.962906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:17.030174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:17.031246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:17.031782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:17.032196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] PATH_INFO: `/v3/users/64e5b41a75354c77a9525faf5f59ce21` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:17.034032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e5b41a75354c77a9525faf5f59ce21)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:17.036107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:17.101136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2406] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:17.113577 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:17.115488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:17.126365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:17.131549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e5b41a75354c77a9525faf5f59ce21'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.142985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e5b41a75354c77a9525faf5f59ce21'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.143693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e5b41a75354c77a9525faf5f59ce21'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.163728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e5b41a75354c77a9525faf5f59ce21'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.172957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e5b41a75354c77a9525faf5f59ce21'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.199849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1222/2407] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:16 2022] DELETE /identity/v3/users/64e5b41a75354c77a9525faf5f59ce21 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:17.214221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc43d76-b1bd-487f-9e8f-5417b2755219 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:17.292632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:17.293503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:17.293997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:17.294355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:17.297201 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:17.383438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:17.385009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d328e-a6ff-46a0-ac55-16bdd2901740 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:17.397172 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2408] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:33:17.573396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:17.575550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:17.627991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260080ee-d85a-4dad-a9f2-96122e9f268d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:17.640696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:17.648529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:17.648529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:17.648529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] PATH_INFO: `/v3/projects/c5b09dda5f624bd884bf2e50c276ddff` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:17.651575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b09dda5f624bd884bf2e50c276ddff)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:17.654222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:17.780000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b09dda5f624bd884bf2e50c276ddff', 'request_id': 'req-427cb560-a526-4c33-8bd6-2926c8c5186a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.790734 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b09dda5f624bd884bf2e50c276ddff', 'request_id': 'req-427cb560-a526-4c33-8bd6-2926c8c5186a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.800233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b09dda5f624bd884bf2e50c276ddff', 'request_id': 'req-427cb560-a526-4c33-8bd6-2926c8c5186a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.817837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] The token cache is being invalidate because project c5b09dda5f624bd884bf2e50c276ddff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:17.818455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.819561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:17.825689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1223/2409] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:17 2022] DELETE /identity/v3/projects/c5b09dda5f624bd884bf2e50c276ddff => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:17.827752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418c3c-4d78-40af-85c4-786d2198f98c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:17.830212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418c3c-4d78-40af-85c4-786d2198f98c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:17.830469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418c3c-4d78-40af-85c4-786d2198f98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:17.854483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:17.856475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:17.868831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-95418c3c-4d78-40af-85c4-786d2198f98c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:17.870085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95418c3c-4d78-40af-85c4-786d2198f98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:17.926146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:17.994387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2410] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:18.004591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.005084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.005522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.039544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cb560-a526-4c33-8bd6-2926c8c5186a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.040039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:18.041834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:18.045409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.046304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.046739 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.047024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] PATH_INFO: `/v3/projects/6a2288a8993743e8a11d244b25bd1fcd` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.048525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2288a8993743e8a11d244b25bd1fcd)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.050468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.058027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2411] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:18.062343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.068637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:18.074269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb75959-c017-4062-9647-3ed1c9d2ede5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.080683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.081433 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.081886 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.082132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.082955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:18.083268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.084960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.104443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2412] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 12:33:18.105515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.116712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b79ea2-395a-46d8-ab4e-320c84fa3367 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.128613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2288a8993743e8a11d244b25bd1fcd', 'request_id': 'req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.142327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2288a8993743e8a11d244b25bd1fcd', 'request_id': 'req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.152476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2288a8993743e8a11d244b25bd1fcd', 'request_id': 'req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.175013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] The token cache is being invalidate because project 6a2288a8993743e8a11d244b25bd1fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:18.175316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.175962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.177536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1224/2413] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:17 2022] DELETE /identity/v3/projects/6a2288a8993743e8a11d244b25bd1fcd => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:33:18.181672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.183166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:18.260455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.261630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.262077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.262516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.264016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.271981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.283789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106928-b2c1-42d1-ab3c-da78bce9c9f9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.290761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.291999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.292603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.293577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] PATH_INFO: `/v3/users/1c7bd6cd08a84a70a52249cb19a0e240` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.308398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7bd6cd08a84a70a52249cb19a0e240)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.310610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.323297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2414] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:18 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:18.327650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.329546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:18.384281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7bd6cd08a84a70a52249cb19a0e240'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.394475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7bd6cd08a84a70a52249cb19a0e240'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.395438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7bd6cd08a84a70a52249cb19a0e240'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.407543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:18.413997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed18ae-a72b-4490-92ea-93b97a6a09da admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.415295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7bd6cd08a84a70a52249cb19a0e240'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.421311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.422036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.423093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.423093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.423866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.425743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7bd6cd08a84a70a52249cb19a0e240'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:18.426203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.436591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1225/2415] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:18 2022] DELETE /identity/v3/users/1c7bd6cd08a84a70a52249cb19a0e240 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:18.458485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2416] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:18 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:33:18.460040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.461882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:18.556028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f66de2-19f7-4a2e-88c6-d4cde176af27 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.564814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.566447 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.566936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.567440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.568410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.570140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.584939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1226/2417] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:33:18.587302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.605392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f7852-773b-449d-b24a-04d2728a2cc3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.616434 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8d7c2-5ffb-4483-80d5-faad56010691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.618084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d7c2-5ffb-4483-80d5-faad56010691 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.618456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d7c2-5ffb-4483-80d5-faad56010691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.618823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d7c2-5ffb-4483-80d5-faad56010691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.620067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8d7c2-5ffb-4483-80d5-faad56010691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.622152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8d7c2-5ffb-4483-80d5-faad56010691 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.637322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2418] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:18.642329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.671082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef5f96-ba14-4e32-ac71-3369b028da67 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.682781 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.683846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.686209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.686611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] PATH_INFO: `/v3/projects/4c3f827fe4dd409fb3e0e29c08851460/users/4da1b68d69954c288796d2d2d66021b2/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.720341 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3f827fe4dd409fb3e0e29c08851460, user_id=4da1b68d69954c288796d2d2d66021b2, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:18.723484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:18.750879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1227/2419] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:18 2022] PUT /identity/v3/projects/4c3f827fe4dd409fb3e0e29c08851460/users/4da1b68d69954c288796d2d2d66021b2/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:18.763963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.764539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.764976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.819003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] MFA Rules not processed for user `4da1b68d69954c288796d2d2d66021b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:18.821070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:18.829744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.832893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:18.898284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:18.950367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218a1c9-6f8b-4c25-9875-f318c0fe7c6d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:18.972334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:18.977035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2420] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:18.978060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:18.979203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:18.979715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:18.980228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] PATH_INFO: `/v3/projects/20a3c7aa55f64f5d88d8d143f080a2d9` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:18.981785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a3c7aa55f64f5d88d8d143f080a2d9)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.003303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.006692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2bd70-7902-41cd-8701-3e8acef36e2c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.011510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.012206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.012481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.012760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.014262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.016941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.080127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2421] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:18 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:19.084589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.089561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:19.092928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a3c7aa55f64f5d88d8d143f080a2d9', 'request_id': 'req-54b3b790-9b49-49aa-88f2-965c411ebe51'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.109828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a3c7aa55f64f5d88d8d143f080a2d9', 'request_id': 'req-54b3b790-9b49-49aa-88f2-965c411ebe51'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.120646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a3c7aa55f64f5d88d8d143f080a2d9', 'request_id': 'req-54b3b790-9b49-49aa-88f2-965c411ebe51'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.137394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] The token cache is being invalidate because project 20a3c7aa55f64f5d88d8d143f080a2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:19.137918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.139129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.141096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1228/2422] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:18 2022] DELETE /identity/v3/projects/20a3c7aa55f64f5d88d8d143f080a2d9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:19.147333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.147333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:19.178573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6830c09-f0e0-42b5-a60d-8902223bbdc9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.184511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.186021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.186317 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.186598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.187347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.189369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.224466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2423] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:19 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:33:19.225788 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.227625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:19.243205 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4715-e812-40b7-bc29-f19abfb4aed7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.250857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.251494 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.252727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.253034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.253944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.256249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.268060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2424] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:33:19.272515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.286057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe4fb3-bd6c-4c99-8b79-6157e7202d9c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.292899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.292899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.293729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.293729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] PATH_INFO: `/v3/projects/8d20d1f1fe734a699f352a1a693cf603/users/35e6790237c241c89daad33776a6aa71/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.313422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d20d1f1fe734a699f352a1a693cf603, user_id=35e6790237c241c89daad33776a6aa71, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.316451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.330996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3b790-9b49-49aa-88f2-965c411ebe51 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.338600 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.339807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.340158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.340631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] PATH_INFO: `/v3/users/c337ca2d26ee45af93f06ffcccd44176` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.342631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2425] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:19 2022] PUT /identity/v3/projects/8d20d1f1fe734a699f352a1a693cf603/users/35e6790237c241c89daad33776a6aa71/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:33:19.346294 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c337ca2d26ee45af93f06ffcccd44176)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.346656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.355619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.434900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c337ca2d26ee45af93f06ffcccd44176'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.445317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c337ca2d26ee45af93f06ffcccd44176'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.446566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c337ca2d26ee45af93f06ffcccd44176'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.451615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd356a0b-94d1-42d7-bbf9-d1326588ad96 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.456590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.457259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.457542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.457816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.458563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.461180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.462094 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c337ca2d26ee45af93f06ffcccd44176'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.470264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-53c57345-1adc-4e28-9518-2bef13b6d775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c337ca2d26ee45af93f06ffcccd44176'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:19.475399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2426] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:33:19.479461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.483378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1229/2427] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:19 2022] DELETE /identity/v3/users/c337ca2d26ee45af93f06ffcccd44176 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:19.484799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:19.504651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e9396-0f50-498e-954c-4fcbf5b53d35 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.518330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:19.585727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.585727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.585727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.585727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] PATH_INFO: `/v3/projects/8d20d1f1fe734a699f352a1a693cf603/users/35e6790237c241c89daad33776a6aa71/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.588247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d20d1f1fe734a699f352a1a693cf603, user_id=35e6790237c241c89daad33776a6aa71, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.590919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.612259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2428] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:19 2022] PUT /identity/v3/projects/8d20d1f1fe734a699f352a1a693cf603/users/35e6790237c241c89daad33776a6aa71/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:19.615620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.615916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.616503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.663038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] MFA Rules not processed for user `35e6790237c241c89daad33776a6aa71`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:19.664868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:19.734564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:19.786875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1230/2429] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:19.811709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.823363 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:19.833808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:19.836639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:19.916939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491538f-39c4-4431-903e-a45b8d4c4934 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.924558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55a1b-fe55-4886-8d6a-c20f08de7abb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.925331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55a1b-fe55-4886-8d6a-c20f08de7abb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.925620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55a1b-fe55-4886-8d6a-c20f08de7abb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.925898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55a1b-fe55-4886-8d6a-c20f08de7abb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.926831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db55ba1-9d87-4459-823c-56def741a57d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:19.932398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:19.933287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:19.933577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:19.934404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] PATH_INFO: `/v3/projects/c979f665603d407eb1d7362ac41d193a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:19.935424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c979f665603d407eb1d7362ac41d193a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.936703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55a1b-fe55-4886-8d6a-c20f08de7abb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:19.937192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.938163 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55a1b-fe55-4886-8d6a-c20f08de7abb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:19.948811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2430] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:33:19.991533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c979f665603d407eb1d7362ac41d193a', 'request_id': 'req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.002433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c979f665603d407eb1d7362ac41d193a', 'request_id': 'req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.011148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c979f665603d407eb1d7362ac41d193a', 'request_id': 'req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.028379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] The token cache is being invalidate because project c979f665603d407eb1d7362ac41d193a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:20.028379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.028379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.030175 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1231/2431] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:19 2022] DELETE /identity/v3/projects/c979f665603d407eb1d7362ac41d193a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:20.035699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db9f56-ef0c-4f73-a075-7dedd5f2cca9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.035992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db9f56-ef0c-4f73-a075-7dedd5f2cca9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.036309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db9f56-ef0c-4f73-a075-7dedd5f2cca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.036479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.037935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:20.054267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-16db9f56-ef0c-4f73-a075-7dedd5f2cca9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:20.055455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16db9f56-ef0c-4f73-a075-7dedd5f2cca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:20.096763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46403ef4-6fa2-4cec-979a-c33c8e6e20c1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.105318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.106173 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.106468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.106946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.109236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:20.120597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2432] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:20.129325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.129619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.129983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.150874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:20.152327 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:20.155750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:20.163667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2433] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:33:20.168731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.179066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29573c3f-0891-4d48-ada6-f9b7897e05a9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.185781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.186474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.186937 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.187055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.187903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:20.188398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.190087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.204849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.209513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2434] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:33:20.209673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.210511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.218903 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1232/2435] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:20.226698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c27c12-b93b-4173-a0f7-d64f51dd98bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.231364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.232121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.232446 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.232729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.233420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.235316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.267309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2436] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:20 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:20.268911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.282895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2b301-0e30-47d7-8dec-3822d11d35b8 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.355962 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.357471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.357967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.358245 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.359018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.361046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.392029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1233/2437] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:20 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:20.397718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.415010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a25848-c619-43f1-bc5d-fe3e1ba8690f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.420546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.421221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.421507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.421785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.422513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.424112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.425761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:20.441379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2438] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:20.442516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.454783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306eb78-8cc4-4a28-8eff-95c9e51e6a30 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.459579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497a547-5fba-4b40-b44d-6a82108ed5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.461791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497a547-5fba-4b40-b44d-6a82108ed5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.461791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497a547-5fba-4b40-b44d-6a82108ed5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.461791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497a547-5fba-4b40-b44d-6a82108ed5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.462866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497a547-5fba-4b40-b44d-6a82108ed5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.464544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497a547-5fba-4b40-b44d-6a82108ed5d6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.477412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1234/2439] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 12:33:20.488614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.501907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406cba3-ba95-41ef-b009-b55782c001f4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.508982 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.509875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.510166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.510499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] PATH_INFO: `/v3/projects/61a0f0a3544e4ddbae81b88f771a2b7f/users/436d3f45a8f541eab413d44e5d264d1f/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.539495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a0f0a3544e4ddbae81b88f771a2b7f, user_id=436d3f45a8f541eab413d44e5d264d1f, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.542340 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.560961 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2440] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:20 2022] PUT /identity/v3/projects/61a0f0a3544e4ddbae81b88f771a2b7f/users/436d3f45a8f541eab413d44e5d264d1f/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:20.572920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.573221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.573522 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.603279 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] MFA Rules not processed for user `436d3f45a8f541eab413d44e5d264d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:20.604485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:20.674668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1235/2441] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:20.686774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.741637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcb3a8-592c-47cf-bea4-85dc798a2ed4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.745710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3840649-c967-432f-8763-fc6272de8769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.746420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840649-c967-432f-8763-fc6272de8769 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.746712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840649-c967-432f-8763-fc6272de8769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.746993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840649-c967-432f-8763-fc6272de8769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.756631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3840649-c967-432f-8763-fc6272de8769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.758060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3840649-c967-432f-8763-fc6272de8769 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.766754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2442] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:33:20.792088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:20.832495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:20.868703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fa6a6-a743-48a5-bf4c-2f454227395a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:20.873649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:20.874795 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.875087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.875369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] PATH_INFO: `/v3/users/436d3f45a8f541eab413d44e5d264d1f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:20.876711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436d3f45a8f541eab413d44e5d264d1f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:20.881111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:20.947334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436d3f45a8f541eab413d44e5d264d1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.956863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436d3f45a8f541eab413d44e5d264d1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.957632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436d3f45a8f541eab413d44e5d264d1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.973837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436d3f45a8f541eab413d44e5d264d1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.981172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436d3f45a8f541eab413d44e5d264d1f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:20.988356 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1236/2443] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:20 2022] DELETE /identity/v3/users/436d3f45a8f541eab413d44e5d264d1f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:20.992990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:20.993585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:20.993701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:21.011805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:21.014782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:21.096084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2444] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:21.106388 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:21.108079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:21.170458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869875c-35a4-4d6a-8e23-924f07195103 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:21.174110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:21.174759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:21.175051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:21.175334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:21.186265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:21.187776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:21.199288 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1237/2445] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:21.511019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:21.512833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:21.526763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139e7c-0af1-4884-a64f-0540d816477b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:21.531731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:21.532650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:21.533389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:21.533389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] PATH_INFO: `/v3/projects/61a0f0a3544e4ddbae81b88f771a2b7f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:21.534407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a0f0a3544e4ddbae81b88f771a2b7f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:21.545550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:21.618997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a0f0a3544e4ddbae81b88f771a2b7f', 'request_id': 'req-85bf2db9-4544-4696-9a70-afad70de162c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:21.633617 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a0f0a3544e4ddbae81b88f771a2b7f', 'request_id': 'req-85bf2db9-4544-4696-9a70-afad70de162c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:21.645879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a0f0a3544e4ddbae81b88f771a2b7f', 'request_id': 'req-85bf2db9-4544-4696-9a70-afad70de162c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:21.659784 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:21.664721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] The token cache is being invalidate because project 61a0f0a3544e4ddbae81b88f771a2b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:21.665259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:21.667400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85bf2db9-4544-4696-9a70-afad70de162c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:21.670896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2446] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:21 2022] DELETE /identity/v3/projects/61a0f0a3544e4ddbae81b88f771a2b7f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:25.329681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:25.331167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:25.344549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032fe95-504b-4750-ad43-e625eaf04050 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:25.402816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:25.411975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:25.412984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:25.413521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:25.413977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] PATH_INFO: `/v3/users/4da1b68d69954c288796d2d2d66021b2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:25.415415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da1b68d69954c288796d2d2d66021b2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:25.417429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:25.475785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da1b68d69954c288796d2d2d66021b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:25.483767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da1b68d69954c288796d2d2d66021b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:25.484694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da1b68d69954c288796d2d2d66021b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:25.499290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da1b68d69954c288796d2d2d66021b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:25.508559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da1b68d69954c288796d2d2d66021b2'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:25.515367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1238/2447] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:25 2022] DELETE /identity/v3/users/4da1b68d69954c288796d2d2d66021b2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:25.520366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:25.520906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:25.521071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:25.539564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:25.540890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:25.610225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2448] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:33:25.627763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:25.630307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:25.693687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b85e7-e2ea-42b5-87ab-07e1dbf65c6f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:25.697432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:25.698122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:25.698411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:25.698692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:25.708525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:25.710021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:25.719663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1239/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:26.005412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:26.007125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:26.020428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf90237-c040-4b9f-a5ac-8c2b9f753ca1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:26.025146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:26.026074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.026424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.026794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] PATH_INFO: `/v3/projects/4c3f827fe4dd409fb3e0e29c08851460` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.028161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3f827fe4dd409fb3e0e29c08851460)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:26.030202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:26.079033 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3f827fe4dd409fb3e0e29c08851460', 'request_id': 'req-903b7675-a6c3-4394-ba65-40d7736b9b07'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.084582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3f827fe4dd409fb3e0e29c08851460', 'request_id': 'req-903b7675-a6c3-4394-ba65-40d7736b9b07'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.092838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3f827fe4dd409fb3e0e29c08851460', 'request_id': 'req-903b7675-a6c3-4394-ba65-40d7736b9b07'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.104525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] The token cache is being invalidate because project 4c3f827fe4dd409fb3e0e29c08851460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:26.104821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.105630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.107338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2450] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:26 2022] DELETE /identity/v3/projects/4c3f827fe4dd409fb3e0e29c08851460 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:26.112067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:26.122045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4780b99-3af6-4231-95fc-f0f3369b4270 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:26.185530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:26.186433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.186712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.186982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] PATH_INFO: `/v3/users/35e6790237c241c89daad33776a6aa71` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.188157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e6790237c241c89daad33776a6aa71)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:26.190134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:26.211314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:26.237836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e6790237c241c89daad33776a6aa71'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.245454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e6790237c241c89daad33776a6aa71'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.246086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e6790237c241c89daad33776a6aa71'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.260259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e6790237c241c89daad33776a6aa71'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.271219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-01097908-8fed-42b3-8aea-6657723238ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e6790237c241c89daad33776a6aa71'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.277373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1240/2451] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:26 2022] DELETE /identity/v3/users/35e6790237c241c89daad33776a6aa71 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:26.564422 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:26.565907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:26.581549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b7675-a6c3-4394-ba65-40d7736b9b07 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:26.646249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:26.647125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.647412 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.647688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] PATH_INFO: `/v3/projects/8d20d1f1fe734a699f352a1a693cf603` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.648951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d20d1f1fe734a699f352a1a693cf603)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:26.650914 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:26.676618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:26.705089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d20d1f1fe734a699f352a1a693cf603', 'request_id': 'req-65576ed1-9464-4a77-9646-b87ef73441f7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.712119 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d20d1f1fe734a699f352a1a693cf603', 'request_id': 'req-65576ed1-9464-4a77-9646-b87ef73441f7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.721884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d20d1f1fe734a699f352a1a693cf603', 'request_id': 'req-65576ed1-9464-4a77-9646-b87ef73441f7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.733797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] The token cache is being invalidate because project 8d20d1f1fe734a699f352a1a693cf603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:26.734189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.734723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-65576ed1-9464-4a77-9646-b87ef73441f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:26.736290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2452] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:26 2022] DELETE /identity/v3/projects/8d20d1f1fe734a699f352a1a693cf603 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:26.741118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.741661 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.741933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.760506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:26.761809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:26.836822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1241/2453] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:33:26.845855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.846160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.846514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.866107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:26.867339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:26.876338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2454] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:33:26.880541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:26.890995 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a121187-b45f-46c6-9d1b-ea24eb6f7455 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:26.895719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:26.896406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.896693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.896974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.897894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:26.898205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:26.900149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:26.916253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1242/2455] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:33:26.917896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:26.927175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27791c2e-c9de-4aa9-9c28-aa16c76136a2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:26.932304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:26.933078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:26.933371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:26.933633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:26.934330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:26.937616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:26.969673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2456] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:26 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:26.970750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:26.982720 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd4dec-db1f-455d-8ff5-cb2b9e69b41a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.049640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.050539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.050893 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.051177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.051899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.054050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.081304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1243/2457] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:26 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:27.082643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.095666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28a000-cc02-4064-a285-d2e262fa078e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.101411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.102164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.102466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.102744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.103651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.105580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.122481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2458] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:27.123561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.141934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90dfe53-249e-4827-84b9-63e6e4db0eda admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.145817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d134d9-7765-4740-bbf0-ac1aba18a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.146541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d134d9-7765-4740-bbf0-ac1aba18a8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.146827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d134d9-7765-4740-bbf0-ac1aba18a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.147297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d134d9-7765-4740-bbf0-ac1aba18a8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.148236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d134d9-7765-4740-bbf0-ac1aba18a8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.150003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d134d9-7765-4740-bbf0-ac1aba18a8fe admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.162414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1244/2459] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:27.167878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.177959 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b262eb-5cd6-4a2f-85d6-06114f1316ed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.183058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.183945 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.184427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.184712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] PATH_INFO: `/v3/projects/b6874a3d5ee541e29bb399ee2a3b5763/users/999b9c3222a349da8dbb66750614b4b4/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.205200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6874a3d5ee541e29bb399ee2a3b5763, user_id=999b9c3222a349da8dbb66750614b4b4, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.207875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.222839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2460] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:27 2022] PUT /identity/v3/projects/b6874a3d5ee541e29bb399ee2a3b5763/users/999b9c3222a349da8dbb66750614b4b4/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:27.240686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f155-031e-47fb-844e-3db446866898 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.240986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f155-031e-47fb-844e-3db446866898 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.241273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f155-031e-47fb-844e-3db446866898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.247356 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:27.274419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-bb73f155-031e-47fb-844e-3db446866898 None None] MFA Rules not processed for user `999b9c3222a349da8dbb66750614b4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:27.275621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bb73f155-031e-47fb-844e-3db446866898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:27.342377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1245/2461] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:27.346885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.411652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a43207-1848-41c1-9184-7bf67ade11b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.416301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.416956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.417240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.417571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.418231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.420816 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.445369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2462] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:27 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:33:27.450026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73f155-031e-47fb-844e-3db446866898 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.461606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73f155-031e-47fb-844e-3db446866898 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.521886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.522556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.522845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.523128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.523823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.525839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.547177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1246/2463] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:27 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:33:27.551343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.561539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29abd-6a4e-4eb9-b293-754f02007139 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.566048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.566744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.567029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.567312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.568029 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.569654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.580942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2464] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:27.584827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.595806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4773a7b-2713-432d-a975-453c5b5c384c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.600754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.601787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.602075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.602353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] PATH_INFO: `/v3/projects/9e661cdf770b4e599d2f6cea95e50752/users/8b9d0abdad444da0b6cdfdb93f6207f4/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.619518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e661cdf770b4e599d2f6cea95e50752, user_id=8b9d0abdad444da0b6cdfdb93f6207f4, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.623017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.637858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1247/2465] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:27 2022] PUT /identity/v3/projects/9e661cdf770b4e599d2f6cea95e50752/users/8b9d0abdad444da0b6cdfdb93f6207f4/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:27.642286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.652310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2bad-d744-439e-adad-23747edbe7fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.670178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:27.718505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798232a-8658-4517-bdd4-ac863dd96727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.719292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798232a-8658-4517-bdd4-ac863dd96727 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.719545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798232a-8658-4517-bdd4-ac863dd96727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.719833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798232a-8658-4517-bdd4-ac863dd96727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.720680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798232a-8658-4517-bdd4-ac863dd96727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.722481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798232a-8658-4517-bdd4-ac863dd96727 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.733567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2466] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:27.737561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.747677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a96f6e-f901-41f9-822d-488c5505c523 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.752543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.753418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.753703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.753977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] PATH_INFO: `/v3/projects/9e661cdf770b4e599d2f6cea95e50752/users/8b9d0abdad444da0b6cdfdb93f6207f4/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.755930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e661cdf770b4e599d2f6cea95e50752, user_id=8b9d0abdad444da0b6cdfdb93f6207f4, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.758638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.769424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1248/2467] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:27 2022] PUT /identity/v3/projects/9e661cdf770b4e599d2f6cea95e50752/users/8b9d0abdad444da0b6cdfdb93f6207f4/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:27.780888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.781186 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.781474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.811151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] MFA Rules not processed for user `8b9d0abdad444da0b6cdfdb93f6207f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:27.812426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:27.879211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2468] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:27.891537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:27.893313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:27.942045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:27.953842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0b905-8fb8-4523-9356-a45b70fa5337 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:27.957632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:27.958289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:27.958574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:27.958855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:27.968393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:27.969839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:27.979585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1249/2469] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:28.109435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:28.121001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224096-ed02-4d84-b2f8-a9829ca2137a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:28.125986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa1f52-87a9-4535-86d5-f503eff5dcde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:28.126699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa1f52-87a9-4535-86d5-f503eff5dcde service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:28.127002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa1f52-87a9-4535-86d5-f503eff5dcde service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:28.127421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa1f52-87a9-4535-86d5-f503eff5dcde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:28.188218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa1f52-87a9-4535-86d5-f503eff5dcde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:28.189694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa1f52-87a9-4535-86d5-f503eff5dcde service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:28.198893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2470] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:28.573815 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:28.652568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb7578-bf4d-41ca-9a37-fb12c5a7a226 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:28.657538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:28.658588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:28.658885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:28.659163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] PATH_INFO: `/v3/users/999b9c3222a349da8dbb66750614b4b4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:28.660421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=999b9c3222a349da8dbb66750614b4b4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:28.662631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:28.735210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999b9c3222a349da8dbb66750614b4b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:28.747515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999b9c3222a349da8dbb66750614b4b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:28.749129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999b9c3222a349da8dbb66750614b4b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:28.774747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999b9c3222a349da8dbb66750614b4b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:28.787497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999b9c3222a349da8dbb66750614b4b4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:28.800272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1250/2471] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:28 2022] DELETE /identity/v3/users/999b9c3222a349da8dbb66750614b4b4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:28.804802 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:28.805618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:28.806080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:28.830587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:28.831943 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:28.881640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:28.903346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2472] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:28.914352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:28.915767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:28.978286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf69df-632b-45e4-a196-5800df7e5652 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:28.982147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:28.982799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:28.983087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:28.983372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:29.001738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:29.003215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:29.018152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1251/2473] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:29.324009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:29.327992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:29.369630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f266bcf5-b1d0-430f-bae7-39e7eb43a85d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:29.376103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:29.376767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:29.377061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:29.377347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] PATH_INFO: `/v3/projects/b6874a3d5ee541e29bb399ee2a3b5763` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:29.378981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6874a3d5ee541e29bb399ee2a3b5763)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:29.382086 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:29.445030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6874a3d5ee541e29bb399ee2a3b5763', 'request_id': 'req-1ec66938-6987-4119-9e8d-4b41081f7de5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.451933 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6874a3d5ee541e29bb399ee2a3b5763', 'request_id': 'req-1ec66938-6987-4119-9e8d-4b41081f7de5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.461948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6874a3d5ee541e29bb399ee2a3b5763', 'request_id': 'req-1ec66938-6987-4119-9e8d-4b41081f7de5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.482165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] The token cache is being invalidate because project b6874a3d5ee541e29bb399ee2a3b5763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:29.482723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.483548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1ec66938-6987-4119-9e8d-4b41081f7de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.486512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2474] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:29 2022] DELETE /identity/v3/projects/b6874a3d5ee541e29bb399ee2a3b5763 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:29.490626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:29.502794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2ac1f-b31e-4c90-a4c9-8c255da5955a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:29.508518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:29.571861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:29.572807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:29.573129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:29.573413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] PATH_INFO: `/v3/users/8b9d0abdad444da0b6cdfdb93f6207f4` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:29.574656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9d0abdad444da0b6cdfdb93f6207f4)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:29.576652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:29.637558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9d0abdad444da0b6cdfdb93f6207f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.646736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9d0abdad444da0b6cdfdb93f6207f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.647526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9d0abdad444da0b6cdfdb93f6207f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.663466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9d0abdad444da0b6cdfdb93f6207f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.672892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9d0abdad444da0b6cdfdb93f6207f4'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:29.679745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1252/2475] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:33:29 2022] DELETE /identity/v3/users/8b9d0abdad444da0b6cdfdb93f6207f4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:29.757564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b8f7-43ae-4315-842b-3fa728886e30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:29.757912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b8f7-43ae-4315-842b-3fa728886e30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:29.758192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b8f7-43ae-4315-842b-3fa728886e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:29.777392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7228b8f7-43ae-4315-842b-3fa728886e30 None None] MFA Rules not processed for user `62766e7afd01401293df7e08707d6d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:29.778789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7228b8f7-43ae-4315-842b-3fa728886e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:29.861897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2476] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:29.976463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:29.983713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:30.056867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382e11-c625-4e7d-8c21-35e9f0424753 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.062500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.063379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.063667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.063948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] PATH_INFO: `/v3/projects/9e661cdf770b4e599d2f6cea95e50752` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.065185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e661cdf770b4e599d2f6cea95e50752)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.067251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.103317 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:30.118851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e661cdf770b4e599d2f6cea95e50752', 'request_id': 'req-07fd2f59-1945-4881-9f06-c34be4b5de7c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:30.133850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e661cdf770b4e599d2f6cea95e50752', 'request_id': 'req-07fd2f59-1945-4881-9f06-c34be4b5de7c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:30.140873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e661cdf770b4e599d2f6cea95e50752', 'request_id': 'req-07fd2f59-1945-4881-9f06-c34be4b5de7c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:30.156258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] The token cache is being invalidate because project 9e661cdf770b4e599d2f6cea95e50752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:33:30.156557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:30.157218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-07fd2f59-1945-4881-9f06-c34be4b5de7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:33:30.158771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1253/2477] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:33:29 2022] DELETE /identity/v3/projects/9e661cdf770b4e599d2f6cea95e50752 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:30.165180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.165476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.165759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.184313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:30.186185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:30.251021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2478] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:30.258607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.258839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.259127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.278768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:30.280008 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:30.288642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1254/2479] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:30.292904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:30.303258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6ca0-06ec-4a80-b309-2a59784ab243 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.307769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.308451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.308741 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.309017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.310316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:33:30.310629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.312241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.318948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:30.328225 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2480] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:33:30.329594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:30.339281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eca199-eece-4f90-beb7-22c88afa68f7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.344596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.345254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.345542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.345821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.346529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.348475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.375051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1255/2481] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:33:30 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:33:30.379603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:30.390729 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccbc8c-5168-4e39-b43a-b243dbf99f21 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.454930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.463496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.463496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.463496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.463496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.463496 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.481162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2482] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:33:30 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:33:30.486107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:30.496558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90391-7ec5-42b3-81dc-36647fd88b2a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.501673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.502332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.502631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.502905 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.503635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.505290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.516090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1256/2483] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:30.520342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:30.531082 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea296961-f638-4c85-a27c-b2835b755ccd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.535743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee145c-8678-409e-a0b4-94157915c6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.536465 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee145c-8678-409e-a0b4-94157915c6bc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.536756 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee145c-8678-409e-a0b4-94157915c6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.537045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee145c-8678-409e-a0b4-94157915c6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.537805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee145c-8678-409e-a0b4-94157915c6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.539401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee145c-8678-409e-a0b4-94157915c6bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.551713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2484] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:33:30.555777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:30.565950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f106d22-a50e-4f48-9548-77eee1d717a8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:30.570548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:30.571441 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.571729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.572010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] PATH_INFO: `/v3/projects/b920d3e4fd0b43f4a54f2cf3f39a5af1/users/3d01dfd10bc449dd80716b20c8212b65/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.587327 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b920d3e4fd0b43f4a54f2cf3f39a5af1, user_id=3d01dfd10bc449dd80716b20c8212b65, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:30.590512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:30.607326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1257/2485] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:33:30 2022] PUT /identity/v3/projects/b920d3e4fd0b43f4a54f2cf3f39a5af1/users/3d01dfd10bc449dd80716b20c8212b65/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:33:30.617475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f52f8f-0f14-40c4-96cf-9dc2f4547d3e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:30.617776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f52f8f-0f14-40c4-96cf-9dc2f4547d3e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:30.618083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f52f8f-0f14-40c4-96cf-9dc2f4547d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:30.645789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-71f52f8f-0f14-40c4-96cf-9dc2f4547d3e None None] MFA Rules not processed for user `3d01dfd10bc449dd80716b20c8212b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:30.647009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71f52f8f-0f14-40c4-96cf-9dc2f4547d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:30.713658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2486] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:31.198683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:31.200828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:31.258803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce4b3d-d648-4eeb-81ae-fb30f18d52b9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:31.262583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:31.263425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:31.263834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:31.264117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:31.274228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:31.275716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:31.286099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1258/2487] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:33:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:33:40.538339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e9724-5e43-4d4b-aa06-c83a4bbf0a30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:40.538811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e9724-5e43-4d4b-aa06-c83a4bbf0a30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:40.539208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e9724-5e43-4d4b-aa06-c83a4bbf0a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:40.559220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-668e9724-5e43-4d4b-aa06-c83a4bbf0a30 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:40.561106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-668e9724-5e43-4d4b-aa06-c83a4bbf0a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:40.617443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2488] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:33:40.632164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:33:40.633608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:40.690089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:40.706814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097c615-5f6c-4bb7-8154-6df753221e47 service glance] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:33:40.711916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c3e2c-44bd-4f0c-9580-1f03d92f791b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:33:40.712595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3e2c-44bd-4f0c-9580-1f03d92f791b service swift] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:40.712882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3e2c-44bd-4f0c-9580-1f03d92f791b service swift] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:40.713162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c3e2c-44bd-4f0c-9580-1f03d92f791b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:40.721255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c3e2c-44bd-4f0c-9580-1f03d92f791b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:33:40.722836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c3e2c-44bd-4f0c-9580-1f03d92f791b service swift] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:33:40.730786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1259/2489] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:33:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 12:33:50.610530 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:50.727540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:33:58.995064 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:33:58.995611 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:33:58.995866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:33:59.038276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] MFA Rules not processed for user `880c0417a58d48ceaab3912957f2e4a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:33:59.039846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:33:59.112149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2490] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:34:06.000425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:06.000425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:06.000918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:06.037519 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:34:06.038964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:06.054930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1260/2491] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jul 19 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:34:06.075093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:06.095111 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229671a-5e4e-45b7-b41f-80bbec4a031b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:06.102437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:06.104017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:06.104618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:06.104907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:06.115720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:06.117450 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:06.130532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2492] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 12:34:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 12:34:07.817948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:07.819979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:07.902088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304956f9-927c-4db6-88b3-b11c09173a74 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:07.907382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:07.908040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:07.908374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:07.908658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:07.910412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:07.922547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:07.924139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:07.934983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1261/2493] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:34:09.299362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:09.300882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:09.369534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653c94-3e55-407b-a091-d96ddb1cb822 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:09.374895 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:09.375770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:09.376049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:09.376397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] PATH_INFO: `/v3/users/f832895e13784dfaa69b44505da8855b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:09.377597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f832895e13784dfaa69b44505da8855b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:09.379526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:09.428222 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f832895e13784dfaa69b44505da8855b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.428893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f832895e13784dfaa69b44505da8855b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.436558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f832895e13784dfaa69b44505da8855b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.443795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f832895e13784dfaa69b44505da8855b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.456202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d7c24951-635f-4595-b815-a69017bc448b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f832895e13784dfaa69b44505da8855b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.462237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2494] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:34:09 2022] DELETE /identity/v3/users/f832895e13784dfaa69b44505da8855b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:34:09.704664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:09.716154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d8714-0577-4681-b8d5-e73a79e4e051 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:09.791515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:09.792481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:09.792770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:09.793565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] PATH_INFO: `/v3/projects/7c5a1bf2e2af4328ab3f1b68b4160b7b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:09.794804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5a1bf2e2af4328ab3f1b68b4160b7b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:09.796803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:09.844253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5a1bf2e2af4328ab3f1b68b4160b7b', 'request_id': 'req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.852298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5a1bf2e2af4328ab3f1b68b4160b7b', 'request_id': 'req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.859527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5a1bf2e2af4328ab3f1b68b4160b7b', 'request_id': 'req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.872476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] The token cache is being invalidate because project 7c5a1bf2e2af4328ab3f1b68b4160b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:34:09.872998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.873833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d40b1614-49cf-4e1f-9a16-c9a5f26a75ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:34:09.876058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1262/2495] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:34:09 2022] DELETE /identity/v3/projects/7c5a1bf2e2af4328ab3f1b68b4160b7b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:34:09.881702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:09.882070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:09.882418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:09.900689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:34:09.901965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:09.907825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:34:10.015336 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2496] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:34:10.018169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.018465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.019051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.050045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:34:10.051142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:10.064890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1263/2497] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:34:10.069793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.080566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecb386-9d3c-4644-8593-b4423f8eb664 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.085885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.086549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.087009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.087514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.088276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:34:10.088588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.090723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.111631 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2498] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:34:10.115642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.129128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f8d9b-083a-420b-848e-e8d08cd3a5cc None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.134158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.134900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.135280 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.135772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.136514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.138621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.142152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:34:10.184438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1264/2499] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:34:10 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 12:34:10.186211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.197415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267e559-9ed2-4ca5-a4b3-aa9f36494a84 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.279514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.280321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.280573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.280876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.283820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.285789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.308125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2500] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:34:10 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:34:10.312788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.330259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dcde6d-c15a-43a3-a5f0-2a9f3de6dc11 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.330259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.330259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.330259 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.330616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.330616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.332085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.343219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1265/2501] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:34:10.347339 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.359454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b25cf8-e2d1-4328-8cfc-be9a634d1f22 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.365000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da815075-e8af-4f58-ad2a-8cc35811f5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.365702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da815075-e8af-4f58-ad2a-8cc35811f5a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.365990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da815075-e8af-4f58-ad2a-8cc35811f5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.366272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da815075-e8af-4f58-ad2a-8cc35811f5a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.367185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da815075-e8af-4f58-ad2a-8cc35811f5a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.368827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da815075-e8af-4f58-ad2a-8cc35811f5a5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.379342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2502] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:34:10.383655 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.394292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671b148-f70e-403a-b3e4-c994d82d73d7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.398884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.399758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.400038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.400346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] PATH_INFO: `/v3/projects/5eea2e8877434a0eb001a4bfd3ffdb15/users/359d2af395ca41dcb8fca434487f4c0b/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.417013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eea2e8877434a0eb001a4bfd3ffdb15, user_id=359d2af395ca41dcb8fca434487f4c0b, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.420406 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.432861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1266/2503] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:34:10 2022] PUT /identity/v3/projects/5eea2e8877434a0eb001a4bfd3ffdb15/users/359d2af395ca41dcb8fca434487f4c0b/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:34:10.442875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.443168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.443448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.476437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] MFA Rules not processed for user `359d2af395ca41dcb8fca434487f4c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:34:10.477696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:10.541498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2504] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:34:10.552455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:10.554126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:10.610079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d9c13-d123-43fa-b4b4-5ff4d89e9358 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:10.613676 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84fcb3-d616-49e8-addb-bdf44dbd0b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:10.614351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84fcb3-d616-49e8-addb-bdf44dbd0b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:10.614636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84fcb3-d616-49e8-addb-bdf44dbd0b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:10.614915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84fcb3-d616-49e8-addb-bdf44dbd0b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:10.623268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84fcb3-d616-49e8-addb-bdf44dbd0b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:10.624999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84fcb3-d616-49e8-addb-bdf44dbd0b89 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:10.633835 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1267/2505] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:34:20.538481 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:34:20.629704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:34:21.067237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:34:21.068671 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:21.124408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72775b-3c6f-4b21-89f6-1a950bd042d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:34:21.128342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:34:21.128986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:34:21.129297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:34:21.129579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:34:21.131321 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:34:21.210320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:34:21.211804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:34:21.221468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2506] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:34:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:34:31.217151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=50 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:35:04.292151 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:35:04.292726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:35:04.292726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:35:04.312055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] MFA Rules not processed for user `1a1284271e83435aa6fd851d6106e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:35:04.313302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:35:04.383007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1268/2507] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:35:14.375972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=25 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:36:11.211196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:36:11.212670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:36:11.225808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8597fb8-f440-46c7-bcac-0492a0bd46f3 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:36:11.229487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:36:11.230126 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:36:11.230411 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:36:11.230715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:36:11.232616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:36:11.290851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:36:11.292313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:36:11.301678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2508] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:36:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:36:19.246331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:36:19.256148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0240-ef64-40c4-a269-3b06c269e4e3 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:36:19.259658 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:36:19.260326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:36:19.260611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:36:19.260887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:36:19.262632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:36:19.332685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:36:19.334123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:36:19.344954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1269/2509] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:36:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:36:21.297125 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:36:22.813815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:36:22.823315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39976d-2fd5-462a-a2e9-fadccdb1cab1 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:36:22.826975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:36:22.827613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:36:22.827896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:36:22.828188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:36:22.830002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:36:22.844000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:36:22.847893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:36:22.859262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2510] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:36:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:36:29.340149 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=34 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:36:32.854373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:37:13.485939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:37:13.495268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cb4c8-71bc-468a-9a74-f326866f720a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:37:13.498874 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:37:13.499772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:37:13.500057 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:37:13.500365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:37:13.502143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:37:13.511585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:37:13.513063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:37:13.521073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1270/2511] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:37:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:37:23.517527 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:37:35.231807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:37:35.233279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:37:35.246847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c9ecb-9554-4770-b4a6-4da82760c882 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:37:35.253044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:37:35.253672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:37:35.253957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:37:35.254235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:37:35.255946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:37:35.265932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:37:35.268152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:37:35.277124 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2512] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:37:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:37:45.272915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:38:12.327541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:38:12.340842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c34a6-fc48-45de-9cd7-cd8ff4301d5a service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:38:12.346308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:38:12.346944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:38:12.347229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:38:12.347508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:38:12.349273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:38:12.408810 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:38:12.410405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:38:12.419207 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1271/2513] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:38:22.063062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:38:22.072319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58df68c-bb61-4751-a366-fc47c5c0ab32 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:38:22.075842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:38:22.076509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:38:22.076799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:38:22.077079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:38:22.079301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:38:22.137938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:38:22.139399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:38:22.150629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2514] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:38:22.415156 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=28 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:38:32.145167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:39:02.695609 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:39:02.714756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcb7ef-009c-499e-a4f5-b1e362da5320 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:39:02.718361 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:39:02.718999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:39:02.719287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:39:02.719567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:39:02.721328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:39:02.803228 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:39:02.804817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:39:02.813583 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1272/2515] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:39:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:39:10.792569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:39:10.794034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:39:10.939242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84a86-8f7a-4367-a0e3-a9c3a2b5a800 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:39:10.944817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:39:10.945455 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:39:10.945739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:39:10.946017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:39:10.947757 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:39:10.959292 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:39:10.960907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:39:10.970452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2516] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:39:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:39:12.809499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=43 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:39:20.966445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=63 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:40:22.327992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:40:22.345943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372e0f3-7a84-4952-a722-5302fa211fe1 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:40:22.349509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:40:22.350140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:40:22.350423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:40:22.350724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:40:22.358902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:40:22.360384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:40:22.369535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1273/2517] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:40:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:40:32.365129 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:41:06.686234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:41:06.687822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:41:06.746850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27c01c-c8cb-482c-999b-d862bd9d2805 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:41:06.754717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:41:06.755535 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:41:06.755864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:41:06.756141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:41:06.758229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:41:06.776206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:41:06.777858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:41:06.791227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2518] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:41:16.785283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=18 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:42:15.479855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:42:15.499479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150ea0c-3144-4ebd-9fef-a2f37a240115 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:42:15.503290 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:42:15.503924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:42:15.504230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:42:15.504509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:42:15.511652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:42:15.513125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:42:15.520978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1274/2519] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:42:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:42:22.739413 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:42:22.749132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4c4b1-ad94-4f03-8daf-2dca0ba0bcd2 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:42:22.752963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:42:22.753591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:42:22.753875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:42:22.754152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:42:22.764825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:42:22.766250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:42:22.776973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2520] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:42:22.822575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:42:22.832761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc181132-6209-4b17-a220-8c0ef57ec851 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:42:22.836805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:42:22.837483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:42:22.837769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:42:22.838047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:42:22.847521 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:42:22.849003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:42:22.880302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1275/2521] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:42:22.940698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:42:22.949733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbe7b8-f00a-43f1-bc73-ea7da5539d5d service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:42:22.954194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:42:22.954928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:42:22.955216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:42:22.955572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:42:22.966969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:42:22.968602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:42:22.988659 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2522] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:42:32.877336 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=23 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:42:32.982906 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=12 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:43:03.316715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:43:03.329714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0e04c-7216-4d56-a6f0-5995cc4dab99 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:43:03.335286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:43:03.335996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:43:03.336394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:43:03.336741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:43:03.345876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:43:03.347352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:43:03.349044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:43:03.355608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99a27a-c277-441d-aa63-a9c8a12600b4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:43:03.359973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:43:03.360837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:43:03.361215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:43:03.361564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:43:03.370094 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:43:03.371528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:43:03.379511 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1276/2523] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:43:03.381399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2524] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:43:12.236357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:43:12.247643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f93cdf-ef11-4438-a2d1-e174cc6f3f0a service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:43:12.253380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:43:12.253976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:43:12.254292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:43:12.254579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:43:12.262376 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:43:12.263960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:43:12.291648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1277/2525] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:43:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:43:13.376855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:43:22.288954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=26 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:43:22.761179 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:43:22.773640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a20df-9baf-405c-a885-2c73eb5ae8eb service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:43:22.779041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:43:22.779679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:43:22.779968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:43:22.780285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:43:22.787994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:43:22.789603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:43:22.817516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2526] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:43:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:43:32.814966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=13 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:44:14.733070 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:44:14.743384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b7dd3-606a-4bfa-914f-dd60ceb3a326 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:44:14.748332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:44:14.748970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:44:14.749257 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:44:14.749535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:44:14.763641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:44:14.765270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:44:14.775659 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1278/2527] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:44:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:44:23.369667 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:44:23.424237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5e99a-637d-4e7f-bb7e-0e7d4017a92d service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:44:23.428247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:44:23.428881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:44:23.429165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:44:23.429443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:44:23.438514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:44:23.439938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:44:23.467775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:44:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:44:24.771394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:44:33.465198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=33 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:46:23.768901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:46:23.768901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:46:23.796791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63440340-40d3-4d9c-ad0d-951878bd8bcb service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:46:23.800677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:46:23.801333 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:46:23.801618 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:46:23.801894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:46:23.803900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:46:23.889877 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:46:23.891337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:46:23.910468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1279/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:46:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:46:33.907786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=46 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:47:21.456699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:47:21.458280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:47:21.524925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bbd2f-7bc7-4ce9-9b1a-7194d8e113a3 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:47:21.528484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:47:21.529118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:47:21.529405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:47:21.529687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:47:21.531540 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:47:21.541137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:47:21.542566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:47:21.551014 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2530] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:47:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:47:23.080162 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:47:23.088575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918c1c2-240a-43cd-80ce-6d3d3e21a40d service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:47:23.092816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:47:23.093445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:47:23.093726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:47:23.094003 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:47:23.095845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:47:23.167958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:47:23.169442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:47:23.178464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1280/2531] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:47:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:47:31.547497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=31 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:47:33.174244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=33 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:48:04.064529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:48:04.073806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fae75-3181-4601-bd7b-8cd7a43e8927 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:48:04.077616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:48:04.078242 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:48:04.078526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:48:04.078806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:48:04.080665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:48:04.150469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:48:04.151931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:48:04.160814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2532] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:48:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:48:11.921630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:48:11.930315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346831f-7b7b-437e-9ffc-d1a9359e5cf5 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:48:11.933931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:48:11.934568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:48:11.934854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:48:11.935134 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:48:11.943348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:48:11.944821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:48:11.953735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1281/2533] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:48:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:48:14.156655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=33 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:48:21.484428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:48:21.493032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97731151-fd3b-452e-918a-af6b025c526a service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:48:21.496668 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40151b9f-5947-4d68-8526-84d7503f56b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:48:21.497313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40151b9f-5947-4d68-8526-84d7503f56b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:48:21.497598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40151b9f-5947-4d68-8526-84d7503f56b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:48:21.497876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40151b9f-5947-4d68-8526-84d7503f56b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:48:21.507350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40151b9f-5947-4d68-8526-84d7503f56b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:48:21.508815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40151b9f-5947-4d68-8526-84d7503f56b4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:48:21.518710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2534] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:48:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:48:21.949644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:48:31.514112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:49:03.730840 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:49:03.732436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:03.827816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0fdfc-1306-4fd6-aab3-9aa45976ab5e service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:49:03.832277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:49:03.833169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:49:03.833456 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:49:03.833736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] PATH_INFO: `/v3/users/359d2af395ca41dcb8fca434487f4c0b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:49:03.834921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=359d2af395ca41dcb8fca434487f4c0b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:49:03.836889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:49:03.881654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '359d2af395ca41dcb8fca434487f4c0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:03.888838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '359d2af395ca41dcb8fca434487f4c0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:03.889645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '359d2af395ca41dcb8fca434487f4c0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:03.902320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '359d2af395ca41dcb8fca434487f4c0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:03.909780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '359d2af395ca41dcb8fca434487f4c0b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:03.915560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1282/2535] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:49:03 2022] DELETE /identity/v3/users/359d2af395ca41dcb8fca434487f4c0b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:49:03.920233 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:49:03.920720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:49:03.920845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:49:03.946137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:49:03.947318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:04.019837 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2536] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:49:04.030668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:49:04.032743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:04.089311 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6cdde-89c3-47ea-9f0d-8bfd84250907 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:49:04.092851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:49:04.093512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:49:04.093793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:49:04.094066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:49:04.100503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:49:04.105539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:49:04.107801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:49:04.117620 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1283/2537] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:49:04.392955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:49:04.394341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:04.407048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f952ee-8f05-4f52-81fe-bb0065384a84 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:49:04.412361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:49:04.413396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:49:04.413688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:49:04.413966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] PATH_INFO: `/v3/projects/5eea2e8877434a0eb001a4bfd3ffdb15` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:49:04.415187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eea2e8877434a0eb001a4bfd3ffdb15)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:49:04.417967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:49:04.466256 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eea2e8877434a0eb001a4bfd3ffdb15', 'request_id': 'req-682f64e8-c816-488d-8ace-79942898071a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:04.471719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eea2e8877434a0eb001a4bfd3ffdb15', 'request_id': 'req-682f64e8-c816-488d-8ace-79942898071a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:04.478916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eea2e8877434a0eb001a4bfd3ffdb15', 'request_id': 'req-682f64e8-c816-488d-8ace-79942898071a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:04.489779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] The token cache is being invalidate because project 5eea2e8877434a0eb001a4bfd3ffdb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:49:04.490073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:04.490712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:49:04.492265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2538] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:49:04 2022] DELETE /identity/v3/projects/5eea2e8877434a0eb001a4bfd3ffdb15 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:49:14.113439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:49:14.486835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:49:20.712372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:49:20.713797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:20.852504 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0692ba-d891-414c-ba0f-c56a6528a219 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:49:20.910964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:49:20.911603 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:49:20.911885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:49:20.912160 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:49:20.914047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:20.925449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:49:20.927017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:49:20.936670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1284/2539] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:49:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:49:24.155219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:49:24.156642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:24.216575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f64e8-c816-488d-8ace-79942898071a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:49:24.220077 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:49:24.220745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:49:24.221028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:49:24.221303 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:49:24.223018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:49:24.292268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:49:24.293689 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:49:24.303155 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2540] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:49:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:49:30.932529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:49:34.299054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=55 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:50:23.268270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:50:23.270186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:50:23.326763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b63a31c-ba18-4226-84d4-243e4a5db0b9 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:50:23.330356 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:50:23.331579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:50:23.331867 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:50:23.332208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:50:23.333951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:50:23.405934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:50:23.407382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:50:23.418235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1285/2541] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:50:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:50:24.614843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:50:24.626021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e3faa-5fe1-4af8-a56a-eb5630018f67 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:50:24.630174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:50:24.630942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:50:24.631288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:50:24.631561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:50:24.633265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:50:24.634440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:50:24.634633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:50:24.653274 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbf49e-d700-425a-96f5-a2fdba1e9bd0 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:50:24.664053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:50:24.664927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:50:24.665146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:50:24.665432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:50:24.665711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:50:24.666392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:50:24.667802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:50:24.681231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2542] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:50:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:50:24.690900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:50:24.693315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:50:24.722019 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1286/2543] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:50:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:50:34.674459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:50:34.702420 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=60 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:52:24.840365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:24.859365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812862c9-ac35-4b9e-a77b-85a3a6307803 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:24.862953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:24.863582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:24.863874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:24.864153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:24.866272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:24.934605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:24.935972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:24.953715 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2544] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:52:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:52:25.392644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:25.394891 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:25.419381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdfcda-b4c3-4ea9-b279-413f010ce361 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:25.423096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:25.423750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:25.424034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:25.424346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:25.426135 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:25.457519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:25.466524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74909d88-35cf-4b8e-91ca-78ba4c275555 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:25.470074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:25.470707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:25.470990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:25.471266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:25.473022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:25.482408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:25.483817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:25.490569 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:25.492272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2545] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:52:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 12:52:25.492663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:25.502692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1287/2546] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:52:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:52:28.558724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:28.560885 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:28.580608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cc281-c4fe-48a6-87bd-b21ae9b2ed49 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:28.586439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:28.587104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:28.587393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:28.587673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:28.589495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:28.604986 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:28.606589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:28.617872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2547] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:52:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:52:32.001541 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:32.013552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7c381-71dc-465d-950a-4172bbce75e1 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:32.018969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:32.019611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:32.019917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:32.020190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:32.021954 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:32.033543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:32.035171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:32.046640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1288/2548] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:52:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:52:32.058108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:32.060157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:32.075674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd81040-dac7-4e21-82c0-f9a6b80a955d service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:32.080221 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:32.081095 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:32.081381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:32.081656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] PATH_INFO: `/v3/users/880c0417a58d48ceaab3912957f2e4a7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:32.095720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=880c0417a58d48ceaab3912957f2e4a7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:32.097670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:32.143026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '880c0417a58d48ceaab3912957f2e4a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.143664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '880c0417a58d48ceaab3912957f2e4a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.151394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '880c0417a58d48ceaab3912957f2e4a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.158470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '880c0417a58d48ceaab3912957f2e4a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.170691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '880c0417a58d48ceaab3912957f2e4a7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.176586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2549] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:52:32 2022] DELETE /identity/v3/users/880c0417a58d48ceaab3912957f2e4a7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:52:32.345667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:32.372596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c6ca4-c0e7-4b4d-a285-e1982b41e126 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:32.467996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:32.469096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:32.469458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:32.469807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] PATH_INFO: `/v3/projects/b4a603ce9a934f188a137ee171d66aa7` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:32.471086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4a603ce9a934f188a137ee171d66aa7)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:32.473387 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:32.529448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4a603ce9a934f188a137ee171d66aa7', 'request_id': 'req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.539170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4a603ce9a934f188a137ee171d66aa7', 'request_id': 'req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.545772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4a603ce9a934f188a137ee171d66aa7', 'request_id': 'req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.556596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] The token cache is being invalidate because project b4a603ce9a934f188a137ee171d66aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:52:32.556987 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.557762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.559373 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1289/2550] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:52:32 2022] DELETE /identity/v3/projects/b4a603ce9a934f188a137ee171d66aa7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:52:32.563316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:32.564884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:32.579021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f938-23b5-4067-ae28-54d3c69d90b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:32.582658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:52:32.637436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:32.638306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:32.638587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:32.639112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] PATH_INFO: `/v3/users/591611d1191e4e5abe9008003cae285a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:32.652731 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591611d1191e4e5abe9008003cae285a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:32.654703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:32.699071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591611d1191e4e5abe9008003cae285a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.699737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591611d1191e4e5abe9008003cae285a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.706818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591611d1191e4e5abe9008003cae285a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.714324 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591611d1191e4e5abe9008003cae285a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.726223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591611d1191e4e5abe9008003cae285a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:32.732110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2551] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:52:32 2022] DELETE /identity/v3/users/591611d1191e4e5abe9008003cae285a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 12:52:32.908844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:52:32.910307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:52:32.917977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:52:32.928085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a94651-1bb3-4f1b-9ad2-02167a98faa5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:52:32.985304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:52:32.986168 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:52:32.986453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:52:32.986728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] PATH_INFO: `/v3/projects/a0b1f6fd65a24bdf84678d6fa552ba22` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:52:32.997892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b1f6fd65a24bdf84678d6fa552ba22)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:52:32.999847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:52:33.050204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b1f6fd65a24bdf84678d6fa552ba22', 'request_id': 'req-9e1b676f-f86d-4a5c-9411-e5169a94013b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:33.058523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b1f6fd65a24bdf84678d6fa552ba22', 'request_id': 'req-9e1b676f-f86d-4a5c-9411-e5169a94013b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:33.065590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b1f6fd65a24bdf84678d6fa552ba22', 'request_id': 'req-9e1b676f-f86d-4a5c-9411-e5169a94013b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:33.076759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] The token cache is being invalidate because project a0b1f6fd65a24bdf84678d6fa552ba22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:52:33.077052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:33.077729 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:52:33.079368 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1290/2552] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:52:32 2022] DELETE /identity/v3/projects/a0b1f6fd65a24bdf84678d6fa552ba22 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:52:42.728575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:52:43.073579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:53:04.133107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:53:04.134590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:04.234761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d447d1-d2fa-4973-8ef1-69be26a0a161 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:53:04.296846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:53:04.297501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:53:04.297793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:53:04.298076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:53:04.299876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:04.366910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:53:04.368591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eece55-5131-43f2-b406-80c1b96707b4 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:53:04.373120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:53:04.387707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2553] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:53:04.517305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:53:04.518713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:04.584419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b676f-f86d-4a5c-9411-e5169a94013b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:53:04.589589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:53:04.590462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:53:04.590752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:53:04.591035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] PATH_INFO: `/v3/users/3d01dfd10bc449dd80716b20c8212b65` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:53:04.592252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d01dfd10bc449dd80716b20c8212b65)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:53:04.594194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:53:04.639970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d01dfd10bc449dd80716b20c8212b65'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:04.647137 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d01dfd10bc449dd80716b20c8212b65'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:04.647760 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d01dfd10bc449dd80716b20c8212b65'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:04.660240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d01dfd10bc449dd80716b20c8212b65'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:04.668552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d01dfd10bc449dd80716b20c8212b65'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:04.674645 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1291/2554] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:53:04 2022] DELETE /identity/v3/users/3d01dfd10bc449dd80716b20c8212b65 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:53:04.679231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:53:04.679538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:53:04.679827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:53:04.696994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:53:04.698199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:04.756440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2555] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:53:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:53:04.766706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:53:04.768086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:04.828009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57527ade-6c7e-49d9-824a-307a7dd0af3d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:53:04.831675 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:53:04.832351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:53:04.832636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:53:04.832911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:53:04.842323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:53:04.843753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:53:04.853006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1292/2556] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:53:05.169407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:53:05.172069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:05.184651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adcbe2-8594-4cf9-bd7d-cf1d8bd9a279 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:53:05.189397 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:53:05.190387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:53:05.190730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:53:05.191020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] PATH_INFO: `/v3/projects/b920d3e4fd0b43f4a54f2cf3f39a5af1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:53:05.192547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b920d3e4fd0b43f4a54f2cf3f39a5af1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:53:05.194418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:53:05.242442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b920d3e4fd0b43f4a54f2cf3f39a5af1', 'request_id': 'req-71367949-2877-49d6-9cdb-d23ec727c1e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:05.248417 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b920d3e4fd0b43f4a54f2cf3f39a5af1', 'request_id': 'req-71367949-2877-49d6-9cdb-d23ec727c1e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:05.255444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b920d3e4fd0b43f4a54f2cf3f39a5af1', 'request_id': 'req-71367949-2877-49d6-9cdb-d23ec727c1e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:05.267677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] The token cache is being invalidate because project b920d3e4fd0b43f4a54f2cf3f39a5af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:53:05.267968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:05.268640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:53:05.270176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2557] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:53:05 2022] DELETE /identity/v3/projects/b920d3e4fd0b43f4a54f2cf3f39a5af1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:53:14.848990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:53:15.264495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:53:21.445666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:53:21.447078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:21.458397 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33466fcd-d72e-481a-8856-7939618fc6f5 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:53:21.508324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:53:21.508968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:53:21.509273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:53:21.509532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:53:21.511466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:53:21.580319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:53:21.581762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:53:21.591079 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1293/2558] 10.222.0.13 () {60 vars in 1396 bytes} [Tue Jul 19 12:53:21 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:53:31.586509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=53 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:54:26.492072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:26.493528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:26.573120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71367949-2877-49d6-9cdb-d23ec727c1e4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:26.578102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:26.578754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:26.579037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:26.579313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:26.581080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:26.594695 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:26.596306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:26.607296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2559] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:30.225270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:30.226721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:30.237178 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1818b9-aa33-4f72-ab7c-142335a0704c service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:30.241244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:30.241431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:30.241712 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:30.241988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:30.243690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:30.309064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:30.310516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:30.320230 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1294/2560] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:33.509968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:33.512237 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:33.543560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ec66e-6bd0-4908-9619-de58fec087e7 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:33.558679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:33.560028 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:33.560498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:33.560929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] PATH_INFO: `/v3/users/62766e7afd01401293df7e08707d6d95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:33.562574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=62766e7afd01401293df7e08707d6d95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:33.564972 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:33.676482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62766e7afd01401293df7e08707d6d95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:33.676973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62766e7afd01401293df7e08707d6d95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:33.688871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62766e7afd01401293df7e08707d6d95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:33.696592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62766e7afd01401293df7e08707d6d95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:33.709252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-97581ac4-87df-4698-8bcd-cd02c229ee9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62766e7afd01401293df7e08707d6d95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:33.715561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2561] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 12:54:33 2022] DELETE /identity/v3/users/62766e7afd01401293df7e08707d6d95 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:54:33.892617 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:33.907890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc164f-2fc7-4ed1-ab9c-22252d922557 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:33.966121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:33.966895 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:33.967183 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:33.967454 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] PATH_INFO: `/v3/projects/12a3207b8cea465d8d1ed5c9e3963a4f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:33.968640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a3207b8cea465d8d1ed5c9e3963a4f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:33.970555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.015799 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a3207b8cea465d8d1ed5c9e3963a4f', 'request_id': 'req-9d27e02b-ed81-4d8f-8951-48803c7fb90a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:34.022630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a3207b8cea465d8d1ed5c9e3963a4f', 'request_id': 'req-9d27e02b-ed81-4d8f-8951-48803c7fb90a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:34.029190 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a3207b8cea465d8d1ed5c9e3963a4f', 'request_id': 'req-9d27e02b-ed81-4d8f-8951-48803c7fb90a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:34.039390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] The token cache is being invalidate because project 12a3207b8cea465d8d1ed5c9e3963a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 12:54:34.039683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:34.040350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-9d27e02b-ed81-4d8f-8951-48803c7fb90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 12:54:34.042363 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1295/2562] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 12:54:33 2022] DELETE /identity/v3/projects/12a3207b8cea465d8d1ed5c9e3963a4f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:54:34.048299 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.048593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.048874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.095129 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:54:34.096294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:34.127605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:54:34.156367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2563] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:54:34.164006 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.164338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.164629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.182535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:54:34.183783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:34.192453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1296/2564] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:54:34.196875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.206942 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dee63e-63fc-4555-a22e-388cc964c0fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.211368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.212041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.212356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.212637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.213979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 12:54:34.214342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.215931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.227153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2565] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 12:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 12:54:34.231711 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.241018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1e56a-a658-498c-925c-cd93a0824a46 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.246221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.246864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.247152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.247429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.248130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.250097 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.263983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:54:34.293312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1297/2566] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 12:54:34 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:54:34.294712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.304967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4e7a0-d2a2-4dd7-b643-5d682ba6ff5e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.361546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.362307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.362595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.362901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.363606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.365597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.390347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2567] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 12:54:34 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 12:54:34.391538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.402390 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7b2e-e01a-4f47-927a-c7832b4f9a09 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.407111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.407771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.408054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.408393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.409115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.410708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.421539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1298/2568] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:54:34.425512 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.436381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512842-6494-45b3-89c2-e335e23eea8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.440956 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3636c-91f3-4e79-ad94-b93e010814f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.441605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3636c-91f3-4e79-ad94-b93e010814f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.441891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3636c-91f3-4e79-ad94-b93e010814f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.442166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3636c-91f3-4e79-ad94-b93e010814f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.442883 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3636c-91f3-4e79-ad94-b93e010814f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.444589 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3636c-91f3-4e79-ad94-b93e010814f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.454421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2569] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 12:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 12:54:34.458761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.469545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04a65-8da2-4ad6-a750-18993395ab8a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.474323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.475213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.475501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.475777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] PATH_INFO: `/v3/projects/c364ae782cdb4e43ad3dd6f4dddc9520/users/93ebf48bf50b455fa85d3e7ad9164a04/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.491493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c364ae782cdb4e43ad3dd6f4dddc9520, user_id=93ebf48bf50b455fa85d3e7ad9164a04, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.494158 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.507705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1299/2570] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 12:54:34 2022] PUT /identity/v3/projects/c364ae782cdb4e43ad3dd6f4dddc9520/users/93ebf48bf50b455fa85d3e7ad9164a04/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 12:54:34.512769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b665ca-2e26-4951-9c2d-6870d56f42ae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.513235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b665ca-2e26-4951-9c2d-6870d56f42ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.513379 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b665ca-2e26-4951-9c2d-6870d56f42ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.533249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e5b665ca-2e26-4951-9c2d-6870d56f42ae None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:54:34.534461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5b665ca-2e26-4951-9c2d-6870d56f42ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:34.595696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2571] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:54:34.606083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:34.607759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:34.664773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9232e7e-2c47-401c-9e57-4c54a95067dc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:34.668440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:34.669083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:34.669366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:34.669643 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:34.679284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:34.680777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:34.690438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1300/2572] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:38.750682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df76df-7dd8-443f-9b09-02fd8e8577d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:38.751158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df76df-7dd8-443f-9b09-02fd8e8577d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:38.751252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df76df-7dd8-443f-9b09-02fd8e8577d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:38.782548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-88df76df-7dd8-443f-9b09-02fd8e8577d7 None None] MFA Rules not processed for user `93ebf48bf50b455fa85d3e7ad9164a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:54:38.783763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-88df76df-7dd8-443f-9b09-02fd8e8577d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:38.834609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:54:38.872235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2573] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:54:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:54:38.891254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:38.892727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:38.982102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f717d9-559c-4db4-b7a4-6786b7b9dab3 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:38.987952 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:38.988646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:38.988931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:38.989208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:38.998075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:38.999837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:39.013205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1301/2574] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:40.381289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:40.381811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:40.381924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:40.400236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] MFA Rules not processed for user `93ebf48bf50b455fa85d3e7ad9164a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:54:40.401988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:40.414050 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2575] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:54:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:54:40.810807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:40.812640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:40.827068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0a99f-fbde-4740-bee9-fe9a8a6913bd service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:40.831417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:40.832433 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:40.832983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:40.833323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:40.836829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:40.853218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:40.854728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:40.868164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1302/2576] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:41.233884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:41.235266 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:41.314498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34438d8a-1ed1-410e-89b4-e3b2ba6f2fcc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:41.319799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:41.320587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:41.320880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:41.321265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:41.323183 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:41.334927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:41.336582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:41.347284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2577] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:49.914992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:54:49.928686 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2aaa9-4531-45b9-99c4-d62bcc45c882 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:54:49.934375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:54:49.935028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:54:49.935315 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:54:49.935598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:54:49.937398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:54:49.950559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:54:49.952147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:54:49.962703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1303/2578] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:54:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:54:51.342911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=47 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:54:59.958339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:55:23.336640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:55:23.346344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520724c9-8d4b-4d97-b822-dc222b323309 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:55:23.349919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:55:23.350554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:55:23.350844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:55:23.351123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:55:23.352882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:55:23.366443 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:55:23.367855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04fd48-2012-4d91-9bd5-28c2a5927c8c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:55:23.378832 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2579] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:55:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:55:24.139592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:55:24.149040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a838b-0ffa-49b6-9cd7-5fdab785cd20 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:55:24.152579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:55:24.153698 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:55:24.153978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:55:24.154253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:55:24.155961 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:55:24.169854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:55:24.171642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:55:24.182371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1304/2580] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:55:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:55:33.373811 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:55:34.177639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:56:26.073277 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:56:26.073807 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:56:26.073916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:56:26.099976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] MFA Rules not processed for user `1a1284271e83435aa6fd851d6106e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:56:26.101206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:56:26.154289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2581] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:56:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 12:56:29.423525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:56:29.427087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:56:29.452468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5b320-f91c-44fa-8ff1-0b72a4a9ccea service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:56:29.459869 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:56:29.460738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:56:29.461126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:56:29.461507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:56:29.463625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:56:29.478104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:56:29.480092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:56:29.506704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1305/2582] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:56:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:56:36.147820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:56:39.487557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:57:05.799254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:57:05.813507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ded387-76b7-4306-88a6-12501cd39470 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:57:05.819394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:57:05.820057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:57:05.820368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:57:05.820678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:57:05.822482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:57:05.835101 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:57:05.836884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:57:05.847188 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2583] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:57:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:57:15.842844 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:57:32.953784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:57:32.962921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e7d46-b352-4824-973e-e78c81e6be59 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:57:32.966396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:57:32.967026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:57:32.967306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:57:32.967580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:57:32.969328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:57:32.990813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:57:32.992788 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f1905-6208-4d49-89cc-5a47d44763f1 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:57:33.001182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1306/2584] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:57:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:57:42.997278 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=11 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:58:14.765588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:58:14.777354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1025c-b9cd-45d7-aabc-eaa2c45e21aa service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:58:14.782790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:58:14.783418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:58:14.783703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:58:14.783983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:58:14.785809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:58:14.808534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:58:14.810104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:58:14.819235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2585] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:58:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:58:20.265423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:58:20.265941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:58:20.266046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:58:20.283819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] MFA Rules not processed for user `1a1284271e83435aa6fd851d6106e3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 12:58:20.285038 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:58:20.292485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1307/2586] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 12:58:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 12:58:22.127598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:58:22.138850 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5eb4d-5cea-48ba-8440-27d49868bc9e service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:58:22.144812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:58:22.145451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:58:22.145745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:58:22.146017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:58:22.147766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:58:22.217039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:58:22.218615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:58:22.238929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2587] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:58:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:58:30.279082 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:58:32.236153 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=47 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 12:59:32.699194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 12:59:32.700599 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:59:32.784876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1975f-7221-45d8-a10d-e8ed2fc3c5db None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 12:59:32.789902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 12:59:32.790543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 12:59:32.790826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 12:59:32.791130 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 12:59:32.792866 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 12:59:32.803772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 12:59:32.805391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 12:59:32.815273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1308/2588] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 12:59:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 12:59:42.811114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=37 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:00:26.084083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:00:26.085521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:00:26.193690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68375a9-ca80-4b35-af7f-86eae8094a9f service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:00:26.197873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:00:26.198527 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:00:26.198804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:00:26.199085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:00:26.209139 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:00:26.211022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:00:26.220419 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2589] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:00:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:00:36.216302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=40 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:02:34.418554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:02:34.437794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136af3-61fa-4cc9-b54b-76bcb9a1cd0c service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:02:34.441404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:02:34.442039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:02:34.442318 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:02:34.442593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:02:34.450492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:02:34.451913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:02:34.459639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1309/2590] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:02:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:02:44.456306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:03:04.651337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:04.660754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4744a-f3ff-4961-8a88-36b0f724893f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:04.664401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c2b9f-d7d7-47b9-9bec-8eff43cdb391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:04.665071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2b9f-d7d7-47b9-9bec-8eff43cdb391 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:04.665360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2b9f-d7d7-47b9-9bec-8eff43cdb391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:04.665644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c2b9f-d7d7-47b9-9bec-8eff43cdb391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:04.683116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c2b9f-d7d7-47b9-9bec-8eff43cdb391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:04.684525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c2b9f-d7d7-47b9-9bec-8eff43cdb391 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:04.705979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2591] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:05.782759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:05.799449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40542a86-fb6f-4829-b359-21f068498f30 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:05.806479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:05.807439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:05.807986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:05.808529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:05.819271 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:05.821416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:05.833785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1310/2592] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:10.276566 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:10.277067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:10.277185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:10.294383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] MFA Rules not processed for user `93ebf48bf50b455fa85d3e7ad9164a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:03:10.295578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:03:10.302355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2593] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:03:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:03:10.606249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:10.615831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c155ba6-57f3-450d-9c13-2a2db0dd7db9 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:10.619526 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:10.620167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:10.620479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:10.620759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:10.628185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:10.629626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:10.638109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1311/2594] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:10.946584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:10.953240 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:10.958705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038427b-3d72-4e0d-b240-7e303c457a8d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:10.962323 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7d03c-4661-4d58-85d0-b1c5c1c2acae service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:10.964325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:10.965049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:10.965334 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:10.965607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:10.965887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:10.966520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:10.966807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:10.967084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:10.974236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:10.975753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:10.977589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:10.979140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:10.985497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2595] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:10.990924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1312/2596] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:03:15.791391 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:15.810652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf740ba-b843-4fc0-8be5-3e10c54b30e9 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:15.814198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:15.814824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:15.815136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:15.815414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:15.824409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:15.825812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:15.851864 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2597] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:16.643558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:16.655806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6909396-26fe-4263-9dbd-3b1a1c71db53 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:16.661974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:16.662619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:16.662907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:16.663184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:16.673071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:16.674657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:16.685081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1313/2598] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:20.216283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:20.228602 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f4a5a-3dd1-45d9-a225-6a5807fa443f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:20.233965 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:20.234597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:20.234881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:20.235160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:20.247891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:20.250513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:20.262149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2599] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:22.215325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:03:22.226547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d587c90-3940-4181-82eb-46b120af75d1 service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:03:22.230861 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88550603-ccc8-4a49-bbdf-697410a58ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:03:22.231913 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88550603-ccc8-4a49-bbdf-697410a58ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:03:22.232394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88550603-ccc8-4a49-bbdf-697410a58ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:03:22.232785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88550603-ccc8-4a49-bbdf-697410a58ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:03:22.243851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88550603-ccc8-4a49-bbdf-697410a58ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:03:22.246319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88550603-ccc8-4a49-bbdf-697410a58ed5 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:03:22.256632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1314/2600] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:03:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:03:30.256866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=45 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:03:32.252314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=30 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:04:38.644838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:04:38.656047 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8c275-27f0-424c-bd12-c69c5eb8e735 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:04:38.661152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:04:38.661781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:04:38.662071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:04:38.662354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:04:38.671296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:04:38.672918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:04:38.683394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2601] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:04:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:04:48.678521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:11.947610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa86af-04ff-4332-8981-79f2a38b218e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:11.948112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa86af-04ff-4332-8981-79f2a38b218e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:11.948248 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa86af-04ff-4332-8981-79f2a38b218e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:11.985863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a6fa86af-04ff-4332-8981-79f2a38b218e None None] MFA Rules not processed for user `70c035662359493e86c58a3d40150eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:05:11.987493 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6fa86af-04ff-4332-8981-79f2a38b218e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:11.994818 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1315/2602] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:05:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:05:12.277010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:12.278637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:12.335440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60dfda-c42d-474f-8529-a1e613ebb2e2 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:12.339208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:12.339852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:12.340138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:12.340448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:12.342206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:12.351681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:12.353202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:12.361586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2603] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:05:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:05:21.983646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=11 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:22.358115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=26 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:25.108049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:25.108049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:25.108784 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:25.180687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:05:25.182080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:25.264804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1316/2604] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 13:05:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:05:25.332264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:25.334501 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:25.455113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798c0a2-4afe-445a-b180-f1a3977d4346 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:25.460032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02f595-920e-4a4e-97fb-eeb04e85d3c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:25.460701 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02f595-920e-4a4e-97fb-eeb04e85d3c5 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:25.460991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02f595-920e-4a4e-97fb-eeb04e85d3c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:25.461270 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02f595-920e-4a4e-97fb-eeb04e85d3c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:25.469192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02f595-920e-4a4e-97fb-eeb04e85d3c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:25.470743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02f595-920e-4a4e-97fb-eeb04e85d3c5 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:25.478630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2605] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 13:05:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 13:05:26.377823 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:26.379389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:26.509932 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1526a-c3e8-4b01-a2d5-c9c875a6cb94 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:26.515876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:26.516579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:26.518403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:26.518690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:26.520713 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:26.589756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:26.633701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:26.635345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a081a-ca20-4a1f-a177-044f2478b885 service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:26.673265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1317/2606] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:05:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:05:35.475439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=41 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:36.666614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=18 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:40.792013 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a7133-5ab3-4ef5-b7d9-e1bd1a5d2918 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:40.792595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a7133-5ab3-4ef5-b7d9-e1bd1a5d2918 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:40.792856 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a7133-5ab3-4ef5-b7d9-e1bd1a5d2918 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:40.794789 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2607] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 13:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 13:05:40.801371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741793a5-4d34-4d65-bd2b-837999d49352 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:40.801813 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741793a5-4d34-4d65-bd2b-837999d49352 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:40.801916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741793a5-4d34-4d65-bd2b-837999d49352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:40.851827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-741793a5-4d34-4d65-bd2b-837999d49352 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:05:40.853136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-741793a5-4d34-4d65-bd2b-837999d49352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:40.864139 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1318/2608] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 13:05:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:05:40.875447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a7133-5ab3-4ef5-b7d9-e1bd1a5d2918 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:40.877235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-428a7133-5ab3-4ef5-b7d9-e1bd1a5d2918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:40.965539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a7133-5ab3-4ef5-b7d9-e1bd1a5d2918 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:40.969010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407060c4-732b-42aa-8341-6124271bff7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:40.969650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407060c4-732b-42aa-8341-6124271bff7c service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:40.969935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407060c4-732b-42aa-8341-6124271bff7c service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:40.970214 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407060c4-732b-42aa-8341-6124271bff7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:40.980062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407060c4-732b-42aa-8341-6124271bff7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:40.982196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407060c4-732b-42aa-8341-6124271bff7c service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:40.992730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2609] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:05:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:05:41.060798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f71e0c-eda9-4dec-b391-ade628d1ce3a None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:41.061087 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f71e0c-eda9-4dec-b391-ade628d1ce3a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:41.061369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f71e0c-eda9-4dec-b391-ade628d1ce3a None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:41.062791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1319/2610] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 13:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 13:05:41.067509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a6cf-9c0c-472f-9006-77a8cdcbf30a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:41.067796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a6cf-9c0c-472f-9006-77a8cdcbf30a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:41.068072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a6cf-9c0c-472f-9006-77a8cdcbf30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:41.107637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0333a6cf-9c0c-472f-9006-77a8cdcbf30a None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:05:41.108787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0333a6cf-9c0c-472f-9006-77a8cdcbf30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:41.115949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2611] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:05:41.127647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f71e0c-eda9-4dec-b391-ade628d1ce3a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:41.139044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f71e0c-eda9-4dec-b391-ade628d1ce3a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:41.142735 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ff271-c9e1-4393-a05e-57777e06645f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:41.143622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ff271-c9e1-4393-a05e-57777e06645f service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:41.143906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ff271-c9e1-4393-a05e-57777e06645f service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:41.144203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ff271-c9e1-4393-a05e-57777e06645f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:41.151666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ff271-c9e1-4393-a05e-57777e06645f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:41.153622 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ff271-c9e1-4393-a05e-57777e06645f service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:41.161909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1320/2612] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:05:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:05:41.244825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970a18-80a0-4fb8-9f80-eb3e1c6e673e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:41.245117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970a18-80a0-4fb8-9f80-eb3e1c6e673e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:41.245396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970a18-80a0-4fb8-9f80-eb3e1c6e673e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:41.246777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2613] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 13:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 13:05:41.252401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd8455-1ca2-425d-978e-16bb4424d4c5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:41.252719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd8455-1ca2-425d-978e-16bb4424d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:41.253027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd8455-1ca2-425d-978e-16bb4424d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:41.272197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-84dd8455-1ca2-425d-978e-16bb4424d4c5 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:05:41.273417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-84dd8455-1ca2-425d-978e-16bb4424d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:41.283405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1321/2614] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:05:41.295813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970a18-80a0-4fb8-9f80-eb3e1c6e673e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:41.297661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a9970a18-80a0-4fb8-9f80-eb3e1c6e673e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:41.311597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970a18-80a0-4fb8-9f80-eb3e1c6e673e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:41.315297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:41.316035 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:41.316358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:41.316635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:41.329312 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:41.330947 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:41.342712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2615] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:05:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:05:44.134536 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2357b05-07e6-4085-9939-d057bde18e71 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:44.135020 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2357b05-07e6-4085-9939-d057bde18e71 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:44.135132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2357b05-07e6-4085-9939-d057bde18e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:44.156157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-c2357b05-07e6-4085-9939-d057bde18e71 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:05:44.157467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c2357b05-07e6-4085-9939-d057bde18e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:05:44.164802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1322/2616] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:05:44.177480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:05:44.193809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958921cf-85a2-47c1-8294-a3f57328a0f4 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:05:44.209543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:05:44.210463 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:05:44.210568 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:05:44.211442 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:05:44.224132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:05:44.225754 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:05:44.241034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2617] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 13:05:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 13:05:54.149748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=25 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:05:54.237713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=56 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:03.820268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:03.820778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:03.820884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:03.839193 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:03.840414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:03.847563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1323/2618] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jul 19 13:06:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:03.861645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:03.873345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcb737-d8cd-45f0-9588-813561c7fd5e service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:03.880338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:03.881100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:03.881583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:03.881966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:03.891653 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:03.893546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:03.901855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2619] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 13:06:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 13:06:13.833076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:13.898458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:19.748830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:19.758118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd929a-ba09-4b91-85dd-c59a95a56c7d None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:19.761644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:19.762286 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:19.762572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:19.762858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:19.764616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:19.850876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:19.852442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:19.871563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1324/2620] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:20.318626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:20.320269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:20.375037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:20.376619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:20.393986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9472d-070f-441b-85e2-648657cea188 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:20.398886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:20.399525 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:20.399806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:20.400078 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] PATH_INFO: `/v3/users/70c035662359493e86c58a3d40150eb8` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:20.401307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c035662359493e86c58a3d40150eb8)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:20.404036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:20.404162 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a5631-0d17-4dd6-8049-28c3cf29e0e6 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:20.409791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:20.410386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:20.410675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:20.410962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:20.412751 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:20.425335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:20.426911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:20.442670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2621] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:20.471571 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c035662359493e86c58a3d40150eb8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.479208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c035662359493e86c58a3d40150eb8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.479980 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c035662359493e86c58a3d40150eb8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.491939 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c035662359493e86c58a3d40150eb8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.499455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7eaa4ecd-3570-4b3c-b9cd-d911b2ce647a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c035662359493e86c58a3d40150eb8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.505532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1325/2622] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:20 2022] DELETE /identity/v3/users/70c035662359493e86c58a3d40150eb8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:20.677716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:20.689067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c9fab-25f2-490c-9fc6-9863623cfeca service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:20.745217 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:20.746563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:20.746845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:20.747121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] PATH_INFO: `/v3/projects/6f6b44bed4904d50b5f1ebfc5ab34743` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:20.748387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6b44bed4904d50b5f1ebfc5ab34743)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:20.750285 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:20.794359 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6b44bed4904d50b5f1ebfc5ab34743', 'request_id': 'req-d44fb812-4720-49af-8bd5-03cdcf7406fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.799818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6b44bed4904d50b5f1ebfc5ab34743', 'request_id': 'req-d44fb812-4720-49af-8bd5-03cdcf7406fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.806273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6b44bed4904d50b5f1ebfc5ab34743', 'request_id': 'req-d44fb812-4720-49af-8bd5-03cdcf7406fb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.817836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] The token cache is being invalidate because project 6f6b44bed4904d50b5f1ebfc5ab34743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:20.818128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.818782 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d44fb812-4720-49af-8bd5-03cdcf7406fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:20.820970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2623] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:20 2022] DELETE /identity/v3/projects/6f6b44bed4904d50b5f1ebfc5ab34743 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:20.826558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:20.827044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:20.827210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:20.872994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:20.874192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:20.883463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:20.935445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1326/2624] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:20.943003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:20.943301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:20.943592 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:20.961490 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:20.962699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:20.970492 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2625] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:20.974540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:20.984756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86ac2f-5351-4251-8c83-4a137384f0e0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:20.989263 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:20.989914 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:20.990198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:20.990481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:20.991208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:20.991518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:20.993154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.005500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1327/2626] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:06:21.009369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:21.018691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb086ce-f4c9-48a9-804d-76ed40e07535 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:21.023211 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:21.023847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.024128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.024435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.025114 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:21.026990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.036354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:21.063055 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2627] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:21 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:21.066967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:21.076663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b01ef-31ff-4a60-a28b-79f96f189f1e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:21.130929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:21.131588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.131870 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.132146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.132923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:21.134832 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.153857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1328/2628] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:21 2022] POST /identity/v3/users => generated 295 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:21.157927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:21.168515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459a4d1-f4b8-4533-a4c0-3ad99a550694 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:21.173006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:21.173648 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.173929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.174206 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.174924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:21.176536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.187474 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2629] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:06:21.191436 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:21.202080 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cc2b3-1594-4e91-b803-330fb47a15a9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:21.206958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e640-afbc-4b1f-851c-2f70aec415ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:21.207649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e640-afbc-4b1f-851c-2f70aec415ae admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.207940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e640-afbc-4b1f-851c-2f70aec415ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.208400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e640-afbc-4b1f-851c-2f70aec415ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.209148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e640-afbc-4b1f-851c-2f70aec415ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:21.210804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e640-afbc-4b1f-851c-2f70aec415ae admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.225761 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1329/2630] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:06:21.227045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:21.238441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d3c85-3e7e-4114-aaec-f09ed6a73226 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:21.243325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:21.244273 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.244565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.244848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] PATH_INFO: `/v3/projects/11d9e30ad3ce440db88579b93d85a4ae/users/27fdfa32565f416d8539a62e7665eeac/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.261521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d9e30ad3ce440db88579b93d85a4ae, user_id=27fdfa32565f416d8539a62e7665eeac, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:21.264352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.281394 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2631] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:21 2022] PUT /identity/v3/projects/11d9e30ad3ce440db88579b93d85a4ae/users/27fdfa32565f416d8539a62e7665eeac/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:21.292059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.293072 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.293353 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.322002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] MFA Rules not processed for user `27fdfa32565f416d8539a62e7665eeac`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:21.323225 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:21.392750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1330/2632] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:21.400569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:21.402806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:21.459621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f3656-7d4e-4cfb-9804-4aa6272d4634 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:21.463305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:21.463955 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:21.464296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:21.464579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:21.472902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:21.474330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:21.483561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2633] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:06:26.752125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:26.753666 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:26.767408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:26.812884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5060ec-86df-488f-b9a5-a22dfae9816a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:26.816467 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:26.817115 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:26.817402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:26.818177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:26.819482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:26.881385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:26.882828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:26.892926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1331/2634] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:30.233174 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:30.234742 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:30.250753 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6186f2-8046-4f13-82c6-513b127e245c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:30.256070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:30.256963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.257276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.257554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] PATH_INFO: `/v3/users/1a1284271e83435aa6fd851d6106e3a8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.258828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1284271e83435aa6fd851d6106e3a8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:30.260767 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:30.308091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1284271e83435aa6fd851d6106e3a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.309316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1284271e83435aa6fd851d6106e3a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.316289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1284271e83435aa6fd851d6106e3a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.323243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1284271e83435aa6fd851d6106e3a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.336288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b505bcec-5832-4fac-bece-6909d6d39e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1284271e83435aa6fd851d6106e3a8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.342469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2635] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:30 2022] DELETE /identity/v3/users/1a1284271e83435aa6fd851d6106e3a8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:30.507085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:30.517829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88384a-250f-412c-ba51-3aba00bf24c0 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:30.575462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:30.576366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.576652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.576930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] PATH_INFO: `/v3/projects/3528f0d517e54016ab254ca0a3c11693` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.578109 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3528f0d517e54016ab254ca0a3c11693)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:30.580084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:30.628978 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3528f0d517e54016ab254ca0a3c11693', 'request_id': 'req-3c4781d7-70f9-4441-bb8f-d59c1a18a054'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.636308 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3528f0d517e54016ab254ca0a3c11693', 'request_id': 'req-3c4781d7-70f9-4441-bb8f-d59c1a18a054'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.642564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3528f0d517e54016ab254ca0a3c11693', 'request_id': 'req-3c4781d7-70f9-4441-bb8f-d59c1a18a054'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.644879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:30.656901 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] The token cache is being invalidate because project 3528f0d517e54016ab254ca0a3c11693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:30.657186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.657830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-3c4781d7-70f9-4441-bb8f-d59c1a18a054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:30.659349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1332/2636] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:30 2022] DELETE /identity/v3/projects/3528f0d517e54016ab254ca0a3c11693 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:30.665072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.665368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.665652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.684449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:30.685709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:30.702996 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:30.748551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2637] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:30.756042 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.756371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.756657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.774900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:30.776140 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:30.784512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1333/2638] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:30.788679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:30.798966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b555c14-4b0b-423d-b1a0-84e904b4c6de None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:30.803559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:30.804244 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.804536 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.804819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.805546 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:30.805862 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:30.807470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:30.826869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2639] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:06:30.827646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:30.838138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012cd5a-e45d-4d7b-becc-8a9c5a1c896c None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:30.842899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:30.843614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.843907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.844216 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.844941 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:30.846909 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:30.877673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1334/2640] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:30 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:30.878798 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:30.890781 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ba239-c654-4187-85d5-e508ab206fd7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:30.954488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:30.955384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:30.955835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:30.956532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:30.957315 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:30.959647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:30.979746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2641] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:30 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:06:30.984434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:30.994847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816de323-e2b4-4f0d-90cc-15c0f734abe5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:30.999369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:30.999974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.000299 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.000577 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.001292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.002886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.013489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1335/2642] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:31.017462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.028046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f1e6-f42c-4ff6-a882-fd86bcb9cac3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.032621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101179-db7e-4532-a6f6-15dec1cf884b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.033264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101179-db7e-4532-a6f6-15dec1cf884b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.033552 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101179-db7e-4532-a6f6-15dec1cf884b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.033831 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101179-db7e-4532-a6f6-15dec1cf884b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.034547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06101179-db7e-4532-a6f6-15dec1cf884b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.036128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06101179-db7e-4532-a6f6-15dec1cf884b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.046865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2643] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:31.050783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.060516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f9f27-4cf7-4caf-a09d-51497d01e7dc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.065031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.065889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.066171 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.066455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] PATH_INFO: `/v3/projects/006f64a4fb24422b804715e4bd080d91/users/34f197f1ea934b8d8f6a6e8a49d2a91f/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.080802 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] RBAC: Authorizing `identity:create_grant(project_id=006f64a4fb24422b804715e4bd080d91, user_id=34f197f1ea934b8d8f6a6e8a49d2a91f, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.083455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.096399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1336/2644] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:31 2022] PUT /identity/v3/projects/006f64a4fb24422b804715e4bd080d91/users/34f197f1ea934b8d8f6a6e8a49d2a91f/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:31.107198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.107487 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.107769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.134622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] MFA Rules not processed for user `34f197f1ea934b8d8f6a6e8a49d2a91f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:31.135834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:31.198048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2645] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:31.202539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.262602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b06eb8-b5ad-4d20-be0e-56467cdf03be admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.267047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.267696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.267982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.268550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.269234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.271135 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.296412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1337/2646] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:31 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:31.300466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.310852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579848dd-c2fa-4f4d-a97a-3bf402be48fb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.323120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:31.370838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.371476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.371760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.372037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.372879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.375106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.396616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2647] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:31 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:31.400428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.411885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83961e9-ce65-435b-8bdd-956c87979fe0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.417045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.417894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.418185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.418501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.419415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.421164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.433148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1338/2648] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:31.437590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.447647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4f928-0680-46b5-9629-ef05180c6a43 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.452967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.453875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.454160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.454437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] PATH_INFO: `/v3/projects/49de1cf2fce443baaf95a902e7cb9bf6/users/666cb82a7b7d4125a3fd67be91caee6e/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.470584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] RBAC: Authorizing `identity:create_grant(project_id=49de1cf2fce443baaf95a902e7cb9bf6, user_id=666cb82a7b7d4125a3fd67be91caee6e, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.473418 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.488966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2649] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:31 2022] PUT /identity/v3/projects/49de1cf2fce443baaf95a902e7cb9bf6/users/666cb82a7b7d4125a3fd67be91caee6e/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:31.492849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.504351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe55bb-9a47-4383-9a22-6e97d1eaabad admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.565316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2d5dd-f634-4b7c-9bdb-bae022307354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.565976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d5dd-f634-4b7c-9bdb-bae022307354 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.566186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d5dd-f634-4b7c-9bdb-bae022307354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.566469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d5dd-f634-4b7c-9bdb-bae022307354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.567194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2d5dd-f634-4b7c-9bdb-bae022307354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.568885 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2d5dd-f634-4b7c-9bdb-bae022307354 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.570936 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:31.580537 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1339/2650] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:31.584686 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.595680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce12e597-2ca0-4d4d-ac08-6bb8baa2939b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.600404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.601288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.601583 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.601865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] PATH_INFO: `/v3/projects/49de1cf2fce443baaf95a902e7cb9bf6/users/666cb82a7b7d4125a3fd67be91caee6e/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.603927 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49de1cf2fce443baaf95a902e7cb9bf6, user_id=666cb82a7b7d4125a3fd67be91caee6e, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.606709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.617356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2651] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:31 2022] PUT /identity/v3/projects/49de1cf2fce443baaf95a902e7cb9bf6/users/666cb82a7b7d4125a3fd67be91caee6e/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:31.629231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.629544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.629829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.656683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] MFA Rules not processed for user `666cb82a7b7d4125a3fd67be91caee6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:31.657894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:31.725900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1340/2652] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:31.736884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:31.738593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:31.795694 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6b836-d639-4689-9e60-ceb9af3d0727 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:31.799287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b5f8d-2069-480b-bbde-9b441591c459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:31.799925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b5f8d-2069-480b-bbde-9b441591c459 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:31.800240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b5f8d-2069-480b-bbde-9b441591c459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:31.800524 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b5f8d-2069-480b-bbde-9b441591c459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:31.809911 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b5f8d-2069-480b-bbde-9b441591c459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:31.811331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b5f8d-2069-480b-bbde-9b441591c459 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:31.820725 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2653] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:33.103695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:33.165547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439325e-2d47-450a-ba9c-933651d4a5ec None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:33.170000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:33.170864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:33.171148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:33.171426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] PATH_INFO: `/v3/users/34f197f1ea934b8d8f6a6e8a49d2a91f` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:33.172637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f197f1ea934b8d8f6a6e8a49d2a91f)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:33.174563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:33.220148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f197f1ea934b8d8f6a6e8a49d2a91f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.227107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f197f1ea934b8d8f6a6e8a49d2a91f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.227724 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f197f1ea934b8d8f6a6e8a49d2a91f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.239663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f197f1ea934b8d8f6a6e8a49d2a91f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.247123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f197f1ea934b8d8f6a6e8a49d2a91f'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.252967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1341/2654] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:33 2022] DELETE /identity/v3/users/34f197f1ea934b8d8f6a6e8a49d2a91f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:33.257428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:33.257888 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:33.258010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:33.278044 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:33.279302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:33.337105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2655] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:33.346868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:33.348329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:33.353853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:33.409632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a94f38-0991-4707-82fd-5e3bbd82f3aa admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:33.413342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:33.413999 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:33.414287 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:33.414566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:33.423991 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:33.425487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:33.435557 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1342/2656] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:33.722348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:33.724533 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:33.746746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c30e5-0eea-4065-ac08-eddcd24a971f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:33.754786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:33.755899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:33.756106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:33.756534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] PATH_INFO: `/v3/projects/006f64a4fb24422b804715e4bd080d91` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:33.757892 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=006f64a4fb24422b804715e4bd080d91)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:33.759870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:33.769195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:33.816480 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '006f64a4fb24422b804715e4bd080d91', 'request_id': 'req-7a30ad3f-ff21-4901-836c-65edfc5d87d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.822995 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '006f64a4fb24422b804715e4bd080d91', 'request_id': 'req-7a30ad3f-ff21-4901-836c-65edfc5d87d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.831306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '006f64a4fb24422b804715e4bd080d91', 'request_id': 'req-7a30ad3f-ff21-4901-836c-65edfc5d87d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.843245 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] The token cache is being invalidate because project 006f64a4fb24422b804715e4bd080d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:33.843469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.844118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.845745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2657] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:33 2022] DELETE /identity/v3/projects/006f64a4fb24422b804715e4bd080d91 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:33.849773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:33.860302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff411605-38ec-4465-b8fc-efd9b408db20 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:33.921029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:33.921906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:33.922188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:33.922463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] PATH_INFO: `/v3/users/666cb82a7b7d4125a3fd67be91caee6e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:33.923793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666cb82a7b7d4125a3fd67be91caee6e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:33.925976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:33.984648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666cb82a7b7d4125a3fd67be91caee6e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.992469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666cb82a7b7d4125a3fd67be91caee6e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:33.993107 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666cb82a7b7d4125a3fd67be91caee6e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.006777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666cb82a7b7d4125a3fd67be91caee6e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.013902 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-ad5efccf-4f26-4744-b591-1a7701edd908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666cb82a7b7d4125a3fd67be91caee6e'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.020464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1343/2658] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:33 2022] DELETE /identity/v3/users/666cb82a7b7d4125a3fd67be91caee6e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:34.294137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.295612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:34.316542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30ad3f-ff21-4901-836c-65edfc5d87d7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.384215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.385493 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.385954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.387210 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] PATH_INFO: `/v3/projects/49de1cf2fce443baaf95a902e7cb9bf6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.389219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49de1cf2fce443baaf95a902e7cb9bf6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.392588 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.452912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49de1cf2fce443baaf95a902e7cb9bf6', 'request_id': 'req-5cfe22be-ae9b-4d1f-8306-14a147124211'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.458368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49de1cf2fce443baaf95a902e7cb9bf6', 'request_id': 'req-5cfe22be-ae9b-4d1f-8306-14a147124211'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.466426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49de1cf2fce443baaf95a902e7cb9bf6', 'request_id': 'req-5cfe22be-ae9b-4d1f-8306-14a147124211'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.477323 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] The token cache is being invalidate because project 49de1cf2fce443baaf95a902e7cb9bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:34.477615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.478268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5cfe22be-ae9b-4d1f-8306-14a147124211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:34.479796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2659] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:34 2022] DELETE /identity/v3/projects/49de1cf2fce443baaf95a902e7cb9bf6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:34.486086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.486601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.486721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.504359 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:34.505568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:34.526125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:34.576063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1344/2660] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:34.584436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.584733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.585016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.605026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:34.606281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:34.615383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2661] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:34.620069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.630631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d293f-d772-4ffd-a3c2-48c306fa0835 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.635789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.636906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.637201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.637478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.638205 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:34.638513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.640197 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.653986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1345/2662] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 13:06:34.658409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.668344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2f513-4520-417d-9dea-90da56898244 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.674198 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.674853 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.675194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.675472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.676170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.678093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.697482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:34.710835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2663] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:34 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:34.713226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.724563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2472d8-7e94-4e7b-acd6-34d896641f6c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.782408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.783084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.783370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.783647 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.784355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.786331 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.806370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1346/2664] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:34 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:06:34.811559 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.822709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c895-a638-4fdd-9378-363175d92468 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.828565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.829232 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.829528 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.829812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.830542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.832146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.842966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2665] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:34.847122 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.858167 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80b444-9240-43b7-bd7e-d6b19423c46e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.866076 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1851f2-6999-4dcc-9d9f-2954bace56b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.866738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1851f2-6999-4dcc-9d9f-2954bace56b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.867021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1851f2-6999-4dcc-9d9f-2954bace56b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.867296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1851f2-6999-4dcc-9d9f-2954bace56b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.868017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1851f2-6999-4dcc-9d9f-2954bace56b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.869817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1851f2-6999-4dcc-9d9f-2954bace56b1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.881642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1347/2666] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:34.885893 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:34.896946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92e2e3-b382-48a5-a40f-12521b9e76ba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:34.901618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:34.902532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.902821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.903104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] PATH_INFO: `/v3/projects/801b76218c2c40dba407e9d1247a942c/users/79b44f4bdbed4f288780026ecd4c2992/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.919479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=801b76218c2c40dba407e9d1247a942c, user_id=79b44f4bdbed4f288780026ecd4c2992, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:34.922190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:34.937070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2667] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:34 2022] PUT /identity/v3/projects/801b76218c2c40dba407e9d1247a942c/users/79b44f4bdbed4f288780026ecd4c2992/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:34.949930 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:34.950218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:34.950500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:34.979411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] MFA Rules not processed for user `79b44f4bdbed4f288780026ecd4c2992`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:34.981882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:35.049147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1348/2668] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:35.053666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.115614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256c5f9-404f-4fe1-9f90-be36992742fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.120200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.121479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.121764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.122039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.122722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.124642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.150460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2669] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:35 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:35.154787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.165604 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c88ffa-0116-4629-b29a-9863e6ad8a90 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.183185 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:35.223854 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.224574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.224859 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.225138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.225821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.227758 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.248472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1349/2670] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:35 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:35.252520 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.262721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06c554-2362-4b89-b79e-186608472561 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.267281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.267938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.268510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.268797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.269522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.271120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.281578 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2671] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:35.285579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.295971 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd930-0941-4f38-8460-480ef4ac5a4a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.300552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.301419 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.301704 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.301981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] PATH_INFO: `/v3/projects/934e1be7154940d2b46039b548ffccee/users/1528e67fa3664d71b0d354f0c430747d/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.316863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=934e1be7154940d2b46039b548ffccee, user_id=1528e67fa3664d71b0d354f0c430747d, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.319513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.331836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1350/2672] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:35 2022] PUT /identity/v3/projects/934e1be7154940d2b46039b548ffccee/users/1528e67fa3664d71b0d354f0c430747d/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:35.335619 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.345678 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae723d-5901-4476-96fd-95f3c605a6c4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.402598 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05746ee-3c47-4389-b4ec-c7b7c60a059a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.403185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05746ee-3c47-4389-b4ec-c7b7c60a059a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.403472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05746ee-3c47-4389-b4ec-c7b7c60a059a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.403766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05746ee-3c47-4389-b4ec-c7b7c60a059a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.404545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05746ee-3c47-4389-b4ec-c7b7c60a059a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.406136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05746ee-3c47-4389-b4ec-c7b7c60a059a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.416204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2673] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:35.421088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.431479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e7a8a-ab9c-4178-a361-fec4661ed7fc admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.436668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.437545 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.437826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.438118 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] PATH_INFO: `/v3/projects/934e1be7154940d2b46039b548ffccee/users/1528e67fa3664d71b0d354f0c430747d/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.440053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=934e1be7154940d2b46039b548ffccee, user_id=1528e67fa3664d71b0d354f0c430747d, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.442785 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.452921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1351/2674] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:35 2022] PUT /identity/v3/projects/934e1be7154940d2b46039b548ffccee/users/1528e67fa3664d71b0d354f0c430747d/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:35.464089 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.464416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.465240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.491304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] MFA Rules not processed for user `1528e67fa3664d71b0d354f0c430747d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:35.492523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:35.511479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:35.559760 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2675] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:35.570875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.572637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:35.632935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e825e-24c2-4b51-ac2a-b910a1f1b72d admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.636612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.637260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.637547 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.637831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.650540 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.652539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.662642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1352/2676] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:35.706674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:35.708257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:35.721236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaac54-22f2-4a31-8315-a4f7d9d42a6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:35.726010 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a118a155-9992-4838-9444-c36a68062001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:35.726815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118a155-9992-4838-9444-c36a68062001 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:35.727164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118a155-9992-4838-9444-c36a68062001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:35.727504 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a118a155-9992-4838-9444-c36a68062001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:35.791801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118a155-9992-4838-9444-c36a68062001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:35.793969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a118a155-9992-4838-9444-c36a68062001 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:35.803440 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2677] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:39.108898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:39.169807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad408-781c-471c-a382-70f59190ad13 service cinder] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:39.174264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:39.175120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:39.175398 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:39.175673 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] PATH_INFO: `/v3/users/79b44f4bdbed4f288780026ecd4c2992` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:39.176849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b44f4bdbed4f288780026ecd4c2992)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:39.178777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:39.194268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:39.223691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b44f4bdbed4f288780026ecd4c2992'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.230165 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b44f4bdbed4f288780026ecd4c2992'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.231136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b44f4bdbed4f288780026ecd4c2992'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.242852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b44f4bdbed4f288780026ecd4c2992'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.250184 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b44f4bdbed4f288780026ecd4c2992'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.256093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1353/2678] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:39 2022] DELETE /identity/v3/users/79b44f4bdbed4f288780026ecd4c2992 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:39.260817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:39.261358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:39.262073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:39.278735 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:39.279926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:39.337247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2679] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:39.346911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:39.348374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:39.406482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff668a-d6f2-414b-9ecb-aa838b5386f5 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:39.410153 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:39.410814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:39.411099 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:39.411380 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:39.420596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:39.422049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:39.431464 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1354/2680] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:39.698579 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:39.700059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:39.715978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc7ce-b395-4aca-8ace-53072893bf5b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:39.736581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:39.737697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:39.737812 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:39.738003 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] PATH_INFO: `/v3/projects/801b76218c2c40dba407e9d1247a942c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:39.739213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=801b76218c2c40dba407e9d1247a942c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:39.741158 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:39.791714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801b76218c2c40dba407e9d1247a942c', 'request_id': 'req-17a225a9-0ebe-44dc-af8f-27bd26035348'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.795023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:39.801081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801b76218c2c40dba407e9d1247a942c', 'request_id': 'req-17a225a9-0ebe-44dc-af8f-27bd26035348'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.810268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801b76218c2c40dba407e9d1247a942c', 'request_id': 'req-17a225a9-0ebe-44dc-af8f-27bd26035348'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.823673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] The token cache is being invalidate because project 801b76218c2c40dba407e9d1247a942c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:39.824034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.824722 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.826639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2681] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:39 2022] DELETE /identity/v3/projects/801b76218c2c40dba407e9d1247a942c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:39.830560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:39.840340 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf3d7-fd02-48b9-bf8d-2fe3a8592883 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:39.896593 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:39.897381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:39.897664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:39.897966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] PATH_INFO: `/v3/users/1528e67fa3664d71b0d354f0c430747d` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:39.899166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1528e67fa3664d71b0d354f0c430747d)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:39.901143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:39.941514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:39.948342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1528e67fa3664d71b0d354f0c430747d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.956890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1528e67fa3664d71b0d354f0c430747d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.957610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1528e67fa3664d71b0d354f0c430747d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.970210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1528e67fa3664d71b0d354f0c430747d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.977602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d22d7df4-49df-42f7-a73d-e91b5c562b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1528e67fa3664d71b0d354f0c430747d'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:39.984375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1355/2682] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:39 2022] DELETE /identity/v3/users/1528e67fa3664d71b0d354f0c430747d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:40.256310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.257795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:40.271714 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a225a9-0ebe-44dc-af8f-27bd26035348 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.330311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.331175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.331462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.331744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] PATH_INFO: `/v3/projects/934e1be7154940d2b46039b548ffccee` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.332951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934e1be7154940d2b46039b548ffccee)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.336387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.380683 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934e1be7154940d2b46039b548ffccee', 'request_id': 'req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:40.386120 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934e1be7154940d2b46039b548ffccee', 'request_id': 'req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:40.393903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934e1be7154940d2b46039b548ffccee', 'request_id': 'req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:40.405330 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] The token cache is being invalidate because project 934e1be7154940d2b46039b548ffccee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:40.405621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:40.406261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ddeb50d8-cce9-4276-a33f-4a9e5d83ced5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:40.407776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2683] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:40 2022] DELETE /identity/v3/projects/934e1be7154940d2b46039b548ffccee => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:40.412637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.413112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.413219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.430568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:40.431871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:40.490304 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1356/2684] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:40.497966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.498257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.498537 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.516740 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:40.517941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:40.525932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2685] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:40.530650 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.540199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce8591c-5d14-4201-81a7-3756acaf829f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.544929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.545588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.545872 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.546147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.546900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:40.547203 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.548853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.561552 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1357/2686] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:06:40.565656 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.574872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1846b-7327-4e60-8a77-ab26e48b66cb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.580058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.580737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.581026 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.581307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.582054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.583987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.611818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2687] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:40.612955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.623416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbd480-b0c8-4cb6-a933-96213c85948f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.678940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.679601 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.679890 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.680187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.680940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.682807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.702864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1358/2688] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:06:40.707092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.717400 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd9c32-a686-4765-83be-a5938799c7d1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.721930 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.722576 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.722863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.723143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.723858 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.725915 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.737291 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2689] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:40.741113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.751616 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc405-f492-40c8-867b-bde7f5933fac admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.756161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178340e5-379b-48fc-bad5-16e51f79645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.756845 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178340e5-379b-48fc-bad5-16e51f79645c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.757132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178340e5-379b-48fc-bad5-16e51f79645c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.757411 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178340e5-379b-48fc-bad5-16e51f79645c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.758194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178340e5-379b-48fc-bad5-16e51f79645c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.759787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178340e5-379b-48fc-bad5-16e51f79645c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.770401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1359/2690] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:40.774246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.784060 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e32d6-b4f5-4359-bc5e-50816fdcf8d6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.788570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.789441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.789727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.790009 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] PATH_INFO: `/v3/projects/cca8684c773e4316b85eb8f354b09791/users/8d3ad704cdd1464d94b2f9fa845fcd35/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.805574 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca8684c773e4316b85eb8f354b09791, user_id=8d3ad704cdd1464d94b2f9fa845fcd35, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.808218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.821926 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2691] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:40 2022] PUT /identity/v3/projects/cca8684c773e4316b85eb8f354b09791/users/8d3ad704cdd1464d94b2f9fa845fcd35/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:40.834120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.834468 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.834786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.861830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] MFA Rules not processed for user `8d3ad704cdd1464d94b2f9fa845fcd35`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:40.863035 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:40.867818 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.869632 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:40.873199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:40.880314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:40.931722 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1360/2692] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:40.936702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.937910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c6e84-9bcf-44da-acdb-3f3343577585 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.942605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.943469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.943749 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.944021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] PATH_INFO: `/v3/users/27fdfa32565f416d8539a62e7665eeac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.945253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fdfa32565f416d8539a62e7665eeac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.946821 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bdcb0-2555-403e-837f-3635d182a2f9 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:40.947200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.952423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:40.953086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:40.953351 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:40.953630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:40.954320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:40.956229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:40.983147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1361/2693] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:06:40.987146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:40.999746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fdfa32565f416d8539a62e7665eeac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.000584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fdfa32565f416d8539a62e7665eeac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.008525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fdfa32565f416d8539a62e7665eeac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.016591 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fdfa32565f416d8539a62e7665eeac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.029136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-321404c1-24e1-4175-9d05-4006bb9051c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fdfa32565f416d8539a62e7665eeac'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.035737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2694] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:40 2022] DELETE /identity/v3/users/27fdfa32565f416d8539a62e7665eeac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:41.040190 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.040499 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.040787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.056221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048596cb-d81d-4102-a7c8-6fd5f5fe81c8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.057780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:41.059020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.112305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.112972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.113255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.113531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.114210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.116126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.118258 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2695] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:41.128702 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.130144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.142483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1362/2696] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:40 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:06:41.144007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.145771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.157258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7c603-7ac0-438f-8dcc-740271fa8938 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.160269 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:41.162933 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.163590 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.163875 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.164148 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.164908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.166702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.177295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1363/2697] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:06:41.181255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.189108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa89c1-4f19-441a-acef-33b82b0c2529 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.191045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41023f6-ee58-42ae-8678-ae1bd481894e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.192666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.193309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.193595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.193876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.195484 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.196378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.196663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.196937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] PATH_INFO: `/v3/projects/e2d0c5e690ce42af9c61cb3af4d5e68c/users/503c480443cf47cf81bfbeb9400bef76/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.202842 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.204296 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.208568 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:41.211926 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d0c5e690ce42af9c61cb3af4d5e68c, user_id=503c480443cf47cf81bfbeb9400bef76, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.214329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2698] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:06:41.215091 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.230418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1364/2699] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:41 2022] PUT /identity/v3/projects/e2d0c5e690ce42af9c61cb3af4d5e68c/users/503c480443cf47cf81bfbeb9400bef76/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:41.234599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.244005 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ed337-8705-48d6-a022-4522755aa95e service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.306878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731920c7-210e-4e0e-9733-f0242df91e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.307662 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731920c7-210e-4e0e-9733-f0242df91e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.307999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731920c7-210e-4e0e-9733-f0242df91e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.308448 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731920c7-210e-4e0e-9733-f0242df91e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.309229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731920c7-210e-4e0e-9733-f0242df91e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.311370 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731920c7-210e-4e0e-9733-f0242df91e68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.322280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2700] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:41.326408 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.337043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702dfde-196e-4982-8e5a-2e292a7126d0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.341668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.342551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.342833 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.343152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] PATH_INFO: `/v3/projects/e2d0c5e690ce42af9c61cb3af4d5e68c/users/503c480443cf47cf81bfbeb9400bef76/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.345157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d0c5e690ce42af9c61cb3af4d5e68c, user_id=503c480443cf47cf81bfbeb9400bef76, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.347838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.367486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1365/2701] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:41 2022] PUT /identity/v3/projects/e2d0c5e690ce42af9c61cb3af4d5e68c/users/503c480443cf47cf81bfbeb9400bef76/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:41.369427 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.369730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.370070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.397805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] MFA Rules not processed for user `503c480443cf47cf81bfbeb9400bef76`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:41.399103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.467627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2702] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:41.479806 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.482635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.492268 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.493696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.548059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:41.555267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a357fa-dd78-480b-ab32-c908a21cba2e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.559517 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2433d4-1bf6-4478-aaaa-dc25fd2026f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.560369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2433d4-1bf6-4478-aaaa-dc25fd2026f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.560656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2433d4-1bf6-4478-aaaa-dc25fd2026f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.560937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2433d4-1bf6-4478-aaaa-dc25fd2026f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.561903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5588ae3-2489-44c4-a0f4-8fafae5d3caf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.566521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.567378 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.567661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.567939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] PATH_INFO: `/v3/projects/11d9e30ad3ce440db88579b93d85a4ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.569165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d9e30ad3ce440db88579b93d85a4ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.570769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2433d4-1bf6-4478-aaaa-dc25fd2026f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.571067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.572437 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2433d4-1bf6-4478-aaaa-dc25fd2026f3 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.582249 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1366/2703] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:41.623072 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d9e30ad3ce440db88579b93d85a4ae', 'request_id': 'req-e0faa60b-c312-4dbb-a856-0a49f030ff68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.629243 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d9e30ad3ce440db88579b93d85a4ae', 'request_id': 'req-e0faa60b-c312-4dbb-a856-0a49f030ff68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.636642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d9e30ad3ce440db88579b93d85a4ae', 'request_id': 'req-e0faa60b-c312-4dbb-a856-0a49f030ff68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.648498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] The token cache is being invalidate because project 11d9e30ad3ce440db88579b93d85a4ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:41.648900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.649720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:41.651254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2704] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:41 2022] DELETE /identity/v3/projects/11d9e30ad3ce440db88579b93d85a4ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:41.657550 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8be0f2-5674-4c38-9220-b68405adaf29 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.657847 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8be0f2-5674-4c38-9220-b68405adaf29 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.658131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8be0f2-5674-4c38-9220-b68405adaf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.678334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9c8be0f2-5674-4c38-9220-b68405adaf29 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:41.679559 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9c8be0f2-5674-4c38-9220-b68405adaf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.682227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.683582 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.742033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1367/2705] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:41.743910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0faa60b-c312-4dbb-a856-0a49f030ff68 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.747769 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.748529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.748815 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.749092 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.749306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.749602 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.749886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.750997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.769132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:41.770717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:41.780556 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1368/2706] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:41.784727 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.793543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45764-1e19-4200-919d-cc84df6c7464 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.798662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.799301 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.799607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.799881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.800631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:41.800940 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.802529 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.813181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.814329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1369/2707] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 13:06:41.814612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.818450 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.823584 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2708] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:41.828670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30faf-16da-4a10-a6c6-12d350a690ff admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.833221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.833856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.834143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.834421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.835096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.837002 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.866892 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1370/2709] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:41 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:41.868041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.878717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be32dca-4d46-4975-b4a8-6e7117853e69 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.911786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:41.937951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.938745 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.939032 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.939307 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.940001 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.941953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:41.963407 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2710] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:41 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:41.968208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:41.981786 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac333c-f85d-4330-af78-8bfa4af3b4ce admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:41.986551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:41.987234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:41.987520 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:41.988010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:41.988775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:41.990372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:42.002814 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1371/2711] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:42.006900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:42.017761 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a856-2436-4ca4-b28f-82f9442a395a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:42.022603 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09845-6aa5-4ba0-b180-fc7a4695c55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:42.023249 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09845-6aa5-4ba0-b180-fc7a4695c55d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:42.023534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09845-6aa5-4ba0-b180-fc7a4695c55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:42.023860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09845-6aa5-4ba0-b180-fc7a4695c55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:42.024871 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09845-6aa5-4ba0-b180-fc7a4695c55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:42.027096 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09845-6aa5-4ba0-b180-fc7a4695c55d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:42.038663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2712] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:42.042708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:42.054391 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefb274-ea73-49d4-ac76-c9c2aea8256f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:42.059416 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:42.060418 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:42.060829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:42.061017 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] PATH_INFO: `/v3/projects/655b99de30324b49b9d33a3f3f4051a8/users/03318d8ab40e4aa5b0db0e893c79ec61/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:42.079894 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655b99de30324b49b9d33a3f3f4051a8, user_id=03318d8ab40e4aa5b0db0e893c79ec61, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:42.082837 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:42.097736 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1372/2713] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:42 2022] PUT /identity/v3/projects/655b99de30324b49b9d33a3f3f4051a8/users/03318d8ab40e4aa5b0db0e893c79ec61/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:42.103128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:42.103423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:42.103709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:42.123708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:42.124978 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:42.188787 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2714] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:42.198344 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:42.200039 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:42.207012 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:42.259966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192da8b-1ebe-4810-a381-e3ed01cfffe6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:42.264123 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b754c7-07da-46d4-b75f-eddb42b21aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:42.264800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b754c7-07da-46d4-b75f-eddb42b21aab service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:42.265084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b754c7-07da-46d4-b75f-eddb42b21aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:42.265364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b754c7-07da-46d4-b75f-eddb42b21aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:42.274414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b754c7-07da-46d4-b75f-eddb42b21aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:42.275849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b754c7-07da-46d4-b75f-eddb42b21aab service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:42.285179 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1373/2715] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:44.498130 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:44.499567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:44.514166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abde37-d4cc-4911-a4cd-3e6bc11cf8c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:44.537031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:44.539254 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:44.540624 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:44.542276 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] PATH_INFO: `/v3/users/8d3ad704cdd1464d94b2f9fa845fcd35` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:44.546415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d3ad704cdd1464d94b2f9fa845fcd35)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:44.549063 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:44.664057 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d3ad704cdd1464d94b2f9fa845fcd35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:44.664929 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d3ad704cdd1464d94b2f9fa845fcd35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:44.676066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d3ad704cdd1464d94b2f9fa845fcd35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:44.688109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d3ad704cdd1464d94b2f9fa845fcd35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:44.706176 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d3ad704cdd1464d94b2f9fa845fcd35'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:44.717974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2716] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:44 2022] DELETE /identity/v3/users/8d3ad704cdd1464d94b2f9fa845fcd35 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:44.720415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:44.720950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:44.721404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:44.761975 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:44.763224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:44.849654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1374/2717] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:44.873744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:44.879454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:44.925934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:44.974688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e43824-cc58-406d-8b08-d54aa1003be4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:44.978565 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:44.979229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:44.979654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:44.979941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:44.996917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:44.998993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:45.011654 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2718] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:45.552723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:45.555544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:45.580198 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f397697-2230-4626-99fc-b2f6e5f4c3f2 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:45.586715 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:45.587876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:45.590471 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:45.590981 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] PATH_INFO: `/v3/projects/cca8684c773e4316b85eb8f354b09791` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:45.592908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca8684c773e4316b85eb8f354b09791)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:45.595531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:45.722250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca8684c773e4316b85eb8f354b09791', 'request_id': 'req-bfa5edea-7156-49a2-9879-3792df44a876'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:45.735253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca8684c773e4316b85eb8f354b09791', 'request_id': 'req-bfa5edea-7156-49a2-9879-3792df44a876'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:45.743332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca8684c773e4316b85eb8f354b09791', 'request_id': 'req-bfa5edea-7156-49a2-9879-3792df44a876'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:45.758054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] The token cache is being invalidate because project cca8684c773e4316b85eb8f354b09791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:45.758792 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:45.759410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:45.764491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1375/2719] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:45 2022] DELETE /identity/v3/projects/cca8684c773e4316b85eb8f354b09791 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:45.765415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:45.777468 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90b643-9081-410a-87ae-7a6da9e82f0b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:45.882041 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:45.883093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:45.890881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:45.891181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] PATH_INFO: `/v3/users/503c480443cf47cf81bfbeb9400bef76` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:45.899128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=503c480443cf47cf81bfbeb9400bef76)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:45.901282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:46.001604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503c480443cf47cf81bfbeb9400bef76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.005414 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503c480443cf47cf81bfbeb9400bef76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.031548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503c480443cf47cf81bfbeb9400bef76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.071281 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503c480443cf47cf81bfbeb9400bef76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.113726 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6427dadf-c302-4217-9037-d99a306ae25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503c480443cf47cf81bfbeb9400bef76'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.132037 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2720] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:45 2022] DELETE /identity/v3/users/503c480443cf47cf81bfbeb9400bef76 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:46.703491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:46.705738 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:46.721678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5edea-7156-49a2-9879-3792df44a876 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:46.757955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:46.789841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:46.790646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:46.790938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:46.791231 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] PATH_INFO: `/v3/projects/e2d0c5e690ce42af9c61cb3af4d5e68c` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:46.793258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d0c5e690ce42af9c61cb3af4d5e68c)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:46.795773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:46.881120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d0c5e690ce42af9c61cb3af4d5e68c', 'request_id': 'req-4042bc0a-b7a7-4799-bdc5-03fab7f10136'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.901844 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d0c5e690ce42af9c61cb3af4d5e68c', 'request_id': 'req-4042bc0a-b7a7-4799-bdc5-03fab7f10136'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.922934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d0c5e690ce42af9c61cb3af4d5e68c', 'request_id': 'req-4042bc0a-b7a7-4799-bdc5-03fab7f10136'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.951204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] The token cache is being invalidate because project e2d0c5e690ce42af9c61cb3af4d5e68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:46.956612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.957532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4042bc0a-b7a7-4799-bdc5-03fab7f10136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:46.959977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1376/2721] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:46 2022] DELETE /identity/v3/projects/e2d0c5e690ce42af9c61cb3af4d5e68c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:46.964732 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:46.965297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:46.965462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.010369 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:47.012304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:47.082932 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:47.115717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2722] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:47.123523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.123922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.124916 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.152011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:47.154690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:47.167400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1377/2723] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:47.172019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.184597 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99566d-e515-4a47-a863-d878dd6eef57 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.194973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.196693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.197151 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.197284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.198024 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:47.198346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.202387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.222865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2724] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:06:47.223996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.239275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf51ee-5237-4c48-82b1-238f68f8de4f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.255853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff24687-8c42-4364-845d-87f282f5f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.256681 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff24687-8c42-4364-845d-87f282f5f286 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.256918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff24687-8c42-4364-845d-87f282f5f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.257394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff24687-8c42-4364-845d-87f282f5f286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.258254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff24687-8c42-4364-845d-87f282f5f286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.260597 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff24687-8c42-4364-845d-87f282f5f286 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.312297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1378/2725] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:47.313382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.324154 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba069fbb-9f05-47ea-96bf-a4b88b3159c5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.412302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.412960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.413448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.431108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.432209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.435386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.435386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.435386 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.436747 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.474488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2726] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:06:47.476620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.482461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] MFA Rules not processed for user `03318d8ab40e4aa5b0db0e893c79ec61`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:47.488710 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01976287-f95a-439c-8c89-399f7956faf1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.494917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:47.495167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.495637 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.495928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.496253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.497951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.500708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.516529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2727] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:47.520896 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.532168 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581810fc-2d76-45d7-aee5-0e0890ac0c9f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.539197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.539919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.540246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.540534 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.541476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.543134 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.564865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2728] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:06:47.568663 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.606879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe7e14-289e-449b-8f9e-c779aaeb3c05 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.614004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28070e-666f-4cb5-ab43-4addb06dc314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.615124 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28070e-666f-4cb5-ab43-4addb06dc314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.615569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28070e-666f-4cb5-ab43-4addb06dc314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.616049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28070e-666f-4cb5-ab43-4addb06dc314 admin admin] PATH_INFO: `/v3/projects/7ff2865ad8bc45d287e1160f88d22361/users/23e9a97b2dc945a4a5097d6f3184ce69/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.646250 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28070e-666f-4cb5-ab43-4addb06dc314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ff2865ad8bc45d287e1160f88d22361, user_id=23e9a97b2dc945a4a5097d6f3184ce69, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.648285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1379/2729] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:47.649219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28070e-666f-4cb5-ab43-4addb06dc314 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.654824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.659073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:47.666087 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2730] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:47 2022] PUT /identity/v3/projects/7ff2865ad8bc45d287e1160f88d22361/users/23e9a97b2dc945a4a5097d6f3184ce69/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:47.677218 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67680fbf-73c0-4be5-9435-829619e4718b None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.677608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.677960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.678253 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.705842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:47.731423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] MFA Rules not processed for user `23e9a97b2dc945a4a5097d6f3184ce69`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:47.734143 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:47.764048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.764797 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.765054 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.765334 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.766043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.769924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.805649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1380/2731] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:47.809365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.849310 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2732] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:47.853472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.866857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101b973-4d54-4148-87e7-c1038a37ad70 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.874860 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:47.908523 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeacc3a-91d6-4dbe-8398-046d26f8f4ae admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.914303 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.914988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.915275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.915555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.916275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.919238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.952223 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1381/2733] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:47.953061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.953918 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.954352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.954991 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.956055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:47.956526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.958629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.968532 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ad2fa-69a2-4776-9112-3cf55aa2e4c7 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:47.974492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:47.975430 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:47.975888 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:47.976401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:47.977302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:47.979150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:47.995023 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2734] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:47.999497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.001273 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1382/2735] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:48.005227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.017576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847b6df-e4c6-42f8-884e-ecd7ea6053ce admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.087313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd60c0-0442-4c1b-94c4-cf6a97d50cdd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.093746 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.094429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.094773 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.095061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.095771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.098034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.112040 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.113366 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.113928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.114268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] PATH_INFO: `/v3/projects/57f6377da3f7422282b53e42aba2be15/users/99b4911b6fcb47fe82fb4eee4758c607/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.123759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2736] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:47 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:48.127977 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.133608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f6377da3f7422282b53e42aba2be15, user_id=99b4911b6fcb47fe82fb4eee4758c607, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.136699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.138897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d296a-9b2c-47e8-ac9f-2399cacf3f97 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.145630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.146561 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.147016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.147554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.148495 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.150325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.167192 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1383/2737] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:48 2022] PUT /identity/v3/projects/57f6377da3f7422282b53e42aba2be15/users/99b4911b6fcb47fe82fb4eee4758c607/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:48.171595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.187081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:48.195690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2738] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:48.201362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.211834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ccb20-8d78-40b9-8584-a26be3730959 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.279442 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab110a-dfe2-4c23-9934-4fefc632ccf2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.284848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.285651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.286024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.286644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.287234 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.289413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.300482 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.301470 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.301737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.302103 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] PATH_INFO: `/v3/projects/be99c6ab9ef941feb1d9190127037af6/users/e58ad034dc4d4a599d9c89e7de24d277/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.306929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1384/2739] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:48.308062 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.321861 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] RBAC: Authorizing `identity:create_grant(project_id=be99c6ab9ef941feb1d9190127037af6, user_id=e58ad034dc4d4a599d9c89e7de24d277, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.322662 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a46760-c086-4e31-8b62-1bf31ffb68a2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.325606 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.329399 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:48.330036 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.331258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.331562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.331750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] PATH_INFO: `/v3/projects/57f6377da3f7422282b53e42aba2be15/users/99b4911b6fcb47fe82fb4eee4758c607/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.334479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f6377da3f7422282b53e42aba2be15, user_id=99b4911b6fcb47fe82fb4eee4758c607, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.337427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.349557 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2740] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:48 2022] PUT /identity/v3/projects/be99c6ab9ef941feb1d9190127037af6/users/e58ad034dc4d4a599d9c89e7de24d277/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:48.353651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.363084 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1385/2741] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:48 2022] PUT /identity/v3/projects/57f6377da3f7422282b53e42aba2be15/users/99b4911b6fcb47fe82fb4eee4758c607/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:48.426863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a645d-de16-4612-8459-0bc84802142a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.508828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4c2b3-bbea-42e0-8966-6f1d9cb8a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.509516 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4c2b3-bbea-42e0-8966-6f1d9cb8a6dd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.510132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4c2b3-bbea-42e0-8966-6f1d9cb8a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.510132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4c2b3-bbea-42e0-8966-6f1d9cb8a6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.510894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4c2b3-bbea-42e0-8966-6f1d9cb8a6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.513460 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4c2b3-bbea-42e0-8966-6f1d9cb8a6dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.525650 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2742] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:06:48.530457 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.545050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9316439-fe3f-4116-b994-ea25e8f77b59 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.551751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.553196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.553566 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.553829 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] PATH_INFO: `/v3/projects/be99c6ab9ef941feb1d9190127037af6/users/e58ad034dc4d4a599d9c89e7de24d277/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.556414 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be99c6ab9ef941feb1d9190127037af6, user_id=e58ad034dc4d4a599d9c89e7de24d277, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.560682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.573671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1386/2743] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:48 2022] PUT /identity/v3/projects/be99c6ab9ef941feb1d9190127037af6/users/e58ad034dc4d4a599d9c89e7de24d277/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:48.583879 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.584227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.584489 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.626878 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] MFA Rules not processed for user `e58ad034dc4d4a599d9c89e7de24d277`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:48.628395 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:48.712208 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2744] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:48.722778 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:48.724929 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:48.803576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779e39-c1b9-442d-ad6b-cfd3755a0fc1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:48.807974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0063d1-938b-46a8-bde7-c2eebc4e42cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:48.808807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0063d1-938b-46a8-bde7-c2eebc4e42cf service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:48.809206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0063d1-938b-46a8-bde7-c2eebc4e42cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:48.809625 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0063d1-938b-46a8-bde7-c2eebc4e42cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:48.903678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0063d1-938b-46a8-bde7-c2eebc4e42cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:48.904843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0063d1-938b-46a8-bde7-c2eebc4e42cf service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:48.915396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1387/2745] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:50.798471 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:50.800030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:50.865519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:50.920375 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad0692-1162-4e04-a98a-17eb49406eaa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:50.925974 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:50.927669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:50.928027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:50.928416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:50.930669 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:51.065364 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:51.068459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:51.088415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2746] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:06:53.505014 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:53.506043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:53.506508 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:53.546816 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] MFA Rules not processed for user `99b4911b6fcb47fe82fb4eee4758c607`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:53.548297 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:53.574110 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:53.719767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1388/2747] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:53.739017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:53.739017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:53.752551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:53.770434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254d86f-3445-43f5-b586-8c713b26ec6e None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:53.775096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafed8bd-6628-47ec-8379-357ef5a4d19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:53.776092 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed8bd-6628-47ec-8379-357ef5a4d19a service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:53.776592 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed8bd-6628-47ec-8379-357ef5a4d19a service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:53.777089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed8bd-6628-47ec-8379-357ef5a4d19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:53.790032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafed8bd-6628-47ec-8379-357ef5a4d19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:53.792499 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafed8bd-6628-47ec-8379-357ef5a4d19a service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:53.821871 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1389/2748] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:06:53.868664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f816ceb-5ef0-4fa6-8a18-9dc097222a43 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:53.875226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed446bf868f940e48c5adb2fbcd479f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:53.877990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:53.877990 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:53.879709 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:54.022908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:54.030106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:54.050423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2749] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:54.228215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:54.228509 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:54.228791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:54.282254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:54.283531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:54.410665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1390/2750] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 13:06:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:54.433785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:54.437022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:54.480461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:54.534691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8ccb-bab2-46fc-b18f-5bdedc415571 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:54.540229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:54.541429 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:54.541758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:54.542040 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:54.552919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:54.554835 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:54.569521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2751] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 13:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 13:06:56.720113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:56.721701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:56.872623 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:56.874251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:56.875737 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1a965-7c70-46f2-9bad-64d55e154a02 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:56.892750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:56.893689 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:56.893977 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:56.894258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:56.898657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:56.900058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434742b-1ea2-4a7c-a035-2cabe2e3228a service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:56.916790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:56.918048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:56.918774 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:56.919969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:56.935805 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1391/2752] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:56.936371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:56.936900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] PATH_INFO: `/v3/users/93ebf48bf50b455fa85d3e7ad9164a04` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:56.943349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ebf48bf50b455fa85d3e7ad9164a04)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:56.945647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.034646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ebf48bf50b455fa85d3e7ad9164a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.035272 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ebf48bf50b455fa85d3e7ad9164a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.061766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ebf48bf50b455fa85d3e7ad9164a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.077571 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ebf48bf50b455fa85d3e7ad9164a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.093184 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-192fd59d-7488-4ea9-a463-e0156bdfd8a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ebf48bf50b455fa85d3e7ad9164a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.108257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2753] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:56 2022] DELETE /identity/v3/users/93ebf48bf50b455fa85d3e7ad9164a04 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:06:57.318544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.330882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f74f-e8d4-4be7-b078-e1ad546d27c5 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:57.389346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:57.413743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:57.414563 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.414850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.415126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] PATH_INFO: `/v3/projects/c364ae782cdb4e43ad3dd6f4dddc9520` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.416426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c364ae782cdb4e43ad3dd6f4dddc9520)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:57.418619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.485896 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c364ae782cdb4e43ad3dd6f4dddc9520', 'request_id': 'req-7501bc22-d755-4e26-b56c-016fb206fbf8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.497573 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c364ae782cdb4e43ad3dd6f4dddc9520', 'request_id': 'req-7501bc22-d755-4e26-b56c-016fb206fbf8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.506611 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c364ae782cdb4e43ad3dd6f4dddc9520', 'request_id': 'req-7501bc22-d755-4e26-b56c-016fb206fbf8'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.519364 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] The token cache is being invalidate because project c364ae782cdb4e43ad3dd6f4dddc9520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:06:57.519651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.520348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-7501bc22-d755-4e26-b56c-016fb206fbf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:57.521911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1392/2754] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:06:57 2022] DELETE /identity/v3/projects/c364ae782cdb4e43ad3dd6f4dddc9520 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:57.526923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.527218 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.527507 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.547874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:57.549195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:57.641733 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2755] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:57.650477 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.650834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.651120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.673595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:57.674753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:57.684514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1393/2756] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:57.688839 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.701556 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed33f9-f0d8-42b8-9921-4eaef4012ea9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:57.706238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:57.706905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.707193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.707473 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.708358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:06:57.708684 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:57.710392 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.726940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2757] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 19 13:06:57.730210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.742204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb2821-992a-494c-943e-a7038e30b909 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:57.747016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:57.747822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.748191 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.748725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.749482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:57.751119 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.780944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1394/2758] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:06:57 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:57.785097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.800215 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54d147-0035-4382-ac2d-1456ffb224e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:57.815354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:57.869458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:57.871901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.871901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.871901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.871901 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:57.873713 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.901645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2759] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:06:57 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:06:57.906050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.917744 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03eb32-82c7-4fb7-8380-e6ccf98cb391 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:57.923252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:57.924071 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.924421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.924787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.925578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:57.927270 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.942682 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1395/2760] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:06:57.946402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.958491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea1c97-f711-46d2-888e-1ae7f5f6dc6d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:57.963213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fedfe-024c-4eda-a6ef-58dac03c3e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:57.963882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fedfe-024c-4eda-a6ef-58dac03c3e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:57.964171 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fedfe-024c-4eda-a6ef-58dac03c3e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:57.964522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fedfe-024c-4eda-a6ef-58dac03c3e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:57.965229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fedfe-024c-4eda-a6ef-58dac03c3e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:57.966833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fedfe-024c-4eda-a6ef-58dac03c3e20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:57.981750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2761] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:06:57.982924 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:57.994716 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67504b0-2b73-40fa-a71d-d72f83b6cd61 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:58.004213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:58.006562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:58.006562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:58.006562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] PATH_INFO: `/v3/projects/491e79a36d5b4f8cb3952be89ae6c03d/users/70ba1847a4044ae6a338cdb4c80af5df/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:58.022619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=491e79a36d5b4f8cb3952be89ae6c03d, user_id=70ba1847a4044ae6a338cdb4c80af5df, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:58.026085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:58.041765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1396/2762] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:06:57 2022] PUT /identity/v3/projects/491e79a36d5b4f8cb3952be89ae6c03d/users/70ba1847a4044ae6a338cdb4c80af5df/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:58.046821 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:58.047202 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:58.047562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:58.066814 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:58.068293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:58.155149 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2763] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:06:58.165629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:58.167610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:58.280608 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ae360-c160-4eca-80d9-5994efd5c8e2 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:58.284931 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7642b-3cfb-4f51-bdf1-41fd9d431587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:58.285846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7642b-3cfb-4f51-bdf1-41fd9d431587 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:58.286332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7642b-3cfb-4f51-bdf1-41fd9d431587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:58.286594 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7642b-3cfb-4f51-bdf1-41fd9d431587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:58.309629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7642b-3cfb-4f51-bdf1-41fd9d431587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:58.315820 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7642b-3cfb-4f51-bdf1-41fd9d431587 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:58.335494 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1397/2764] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:06:59.392711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:59.396823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:59.424881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae8daa-bd3e-4437-94a2-4e04f1aa8d6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:59.438216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:59.439567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:59.440262 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:59.440803 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] PATH_INFO: `/v3/users/23e9a97b2dc945a4a5097d6f3184ce69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:59.442301 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e9a97b2dc945a4a5097d6f3184ce69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:59.444791 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:59.535075 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e9a97b2dc945a4a5097d6f3184ce69'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:59.536124 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e9a97b2dc945a4a5097d6f3184ce69'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:59.552572 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e9a97b2dc945a4a5097d6f3184ce69'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:59.563185 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e9a97b2dc945a4a5097d6f3184ce69'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:59.580390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e9a97b2dc945a4a5097d6f3184ce69'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:06:59.588017 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2765] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:06:59 2022] DELETE /identity/v3/users/23e9a97b2dc945a4a5097d6f3184ce69 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:06:59.592672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:59.594244 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:59.594762 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:59.634337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:06:59.637868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:59.643900 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:59.766996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1398/2766] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:06:59.779786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:06:59.781252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:06:59.800680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:06:59.872113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9cd3d-af8b-40cc-93d7-9e74255ceeed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:06:59.877903 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:06:59.879062 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:06:59.879575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:06:59.879969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:06:59.901192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:06:59.905497 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:06:59.926944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2767] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:00.493579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:00.495976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:00.526180 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c66f68-6bcd-4a9b-aba4-41f6ece8b3c1 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:00.537239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:00.539979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:00.541051 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:00.541635 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] PATH_INFO: `/v3/projects/7ff2865ad8bc45d287e1160f88d22361` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:00.546516 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ff2865ad8bc45d287e1160f88d22361)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:00.564528 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:00.653812 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ff2865ad8bc45d287e1160f88d22361', 'request_id': 'req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.668629 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ff2865ad8bc45d287e1160f88d22361', 'request_id': 'req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.682575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ff2865ad8bc45d287e1160f88d22361', 'request_id': 'req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.696479 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] The token cache is being invalidate because project 7ff2865ad8bc45d287e1160f88d22361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:07:00.697016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.697855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.700789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1399/2768] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:07:00 2022] DELETE /identity/v3/projects/7ff2865ad8bc45d287e1160f88d22361 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:00.705477 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:00.718030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3904ab-0e7e-4554-a5d8-1a483082fb18 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:00.846880 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:00.848294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:00.848594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:00.848882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] PATH_INFO: `/v3/users/e58ad034dc4d4a599d9c89e7de24d277` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:00.853123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58ad034dc4d4a599d9c89e7de24d277)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:00.855898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:00.968988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58ad034dc4d4a599d9c89e7de24d277'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.969677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58ad034dc4d4a599d9c89e7de24d277'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:00.990083 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58ad034dc4d4a599d9c89e7de24d277'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.001696 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58ad034dc4d4a599d9c89e7de24d277'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.026968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-eced855c-af9c-4f37-bd8c-dc8e71cc508b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58ad034dc4d4a599d9c89e7de24d277'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.037311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2769] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:07:00 2022] DELETE /identity/v3/users/e58ad034dc4d4a599d9c89e7de24d277 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:01.499507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:01.501413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:01.535850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d9b0-9992-4cbc-9206-1ae3c33be26c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:01.662319 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:01.671606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:01.672172 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:01.673277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] PATH_INFO: `/v3/projects/be99c6ab9ef941feb1d9190127037af6` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:01.675227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=be99c6ab9ef941feb1d9190127037af6)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:01.677801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:01.770606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:01.790402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be99c6ab9ef941feb1d9190127037af6', 'request_id': 'req-d0bd137e-cb9f-42e9-ad81-cc8296744bad'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.809232 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be99c6ab9ef941feb1d9190127037af6', 'request_id': 'req-d0bd137e-cb9f-42e9-ad81-cc8296744bad'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.823490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be99c6ab9ef941feb1d9190127037af6', 'request_id': 'req-d0bd137e-cb9f-42e9-ad81-cc8296744bad'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.860004 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] The token cache is being invalidate because project be99c6ab9ef941feb1d9190127037af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:07:01.860396 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.862997 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d0bd137e-cb9f-42e9-ad81-cc8296744bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:01.871382 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1400/2770] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:07:01 2022] DELETE /identity/v3/projects/be99c6ab9ef941feb1d9190127037af6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:01.878687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:01.879652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:01.880318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:01.925615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:01.931981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:02.001278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:02.086881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2771] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:02.096166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360733f-c606-4389-8467-c694784aafa6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.096475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360733f-c606-4389-8467-c694784aafa6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.096767 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360733f-c606-4389-8467-c694784aafa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.141692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4360733f-c606-4389-8467-c694784aafa6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:02.142960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4360733f-c606-4389-8467-c694784aafa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:02.161181 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:02.173472 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1401/2772] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:02.174770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a227dd2-8e0e-446f-a0ef-5b8bfa23f74a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:02.184941 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:02.188639 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.189076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.189430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.191641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:07:02.192034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:02.193806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:02.219830 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360733f-c606-4389-8467-c694784aafa6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:02.220048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2773] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:07:02.237572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360733f-c606-4389-8467-c694784aafa6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:02.256576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:02.263199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.263551 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.264768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.266352 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:02.271289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:02.345500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1402/2774] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:07:02 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:07:02.360898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:02.379951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c51e3-1b28-4929-b981-72e7f7b0a632 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:02.503347 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:02.504302 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.504770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.505228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.506116 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:02.508286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:02.564251 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2775] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:07:02 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:07:02.565426 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:02.586907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250fa54-5269-448b-81f1-dc5a6f9cf2b8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:02.602210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:02.603489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.603489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.603489 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.604989 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:02.608276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:02.648113 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1403/2776] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:07:02.651997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:02.664825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1ebbb-99f1-4cc2-b00f-2a765d881679 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:02.685727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5f29f-720e-4164-a320-8e80d4574bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:02.686423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f29f-720e-4164-a320-8e80d4574bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.686716 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f29f-720e-4164-a320-8e80d4574bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.686999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5f29f-720e-4164-a320-8e80d4574bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.687869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f29f-720e-4164-a320-8e80d4574bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:02.689707 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5f29f-720e-4164-a320-8e80d4574bfe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:02.717962 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2777] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:07:02.719077 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:02.736075 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9fd10c-2daf-4144-89c7-cb67937b099c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:02.762405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:02.763405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.763700 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.763979 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] PATH_INFO: `/v3/projects/d509204b7b144985be48fbfc93d14297/users/e6426ed3bbac4deba9b202d131c74f05/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.832141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d509204b7b144985be48fbfc93d14297, user_id=e6426ed3bbac4deba9b202d131c74f05, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:02.834935 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:02.870938 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1404/2778] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:07:02 2022] PUT /identity/v3/projects/d509204b7b144985be48fbfc93d14297/users/e6426ed3bbac4deba9b202d131c74f05/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:07:02.883792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:02.884625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:02.885076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:02.953019 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] MFA Rules not processed for user `e6426ed3bbac4deba9b202d131c74f05`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:02.954563 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:03.154329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2779] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:03.155227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:03.305953 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8956fd-7418-48c8-acc6-86db91666dc8 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:03.320780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:03.321631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:03.322021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:03.322410 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:03.323314 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:03.329444 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:03.383007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1405/2780] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:07:03 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:07:03.391456 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:03.407652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222051c-57c2-4c1d-a4eb-0952a3a9b396 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:03.448521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:03.488973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:03.494595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:03.494957 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:03.496875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:03.496875 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:03.501081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:03.546046 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:03.546469 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2781] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:07:03 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:07:03.563960 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034becac-2cc9-4786-91a2-9dff4d9722ce admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:03.577116 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:03.577822 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:03.578112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:03.578539 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:03.579117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:03.580983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:03.598417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1406/2782] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:07:03.608362 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:03.617628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e62fb-775d-4c27-8d32-94937c20930e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:03.633445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:03.634371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:03.634658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:03.634939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] PATH_INFO: `/v3/projects/75b40e3300ca47d99f4161af179ccb7e/users/cb2dfa0faf5044f5b32edd05b97d7719/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:03.654723 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b40e3300ca47d99f4161af179ccb7e, user_id=cb2dfa0faf5044f5b32edd05b97d7719, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:03.657220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:03.690081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2783] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:07:03 2022] PUT /identity/v3/projects/75b40e3300ca47d99f4161af179ccb7e/users/cb2dfa0faf5044f5b32edd05b97d7719/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:03.693984 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:03.705920 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00bf5-a116-4049-8273-b6f9dfb7649a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:03.781044 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:03.854506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efc9df-6e2c-48e3-a596-0d8cc28af5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:03.855409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efc9df-6e2c-48e3-a596-0d8cc28af5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:03.856320 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efc9df-6e2c-48e3-a596-0d8cc28af5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:03.856679 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efc9df-6e2c-48e3-a596-0d8cc28af5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:03.858548 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efc9df-6e2c-48e3-a596-0d8cc28af5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:03.861283 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efc9df-6e2c-48e3-a596-0d8cc28af5a2 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:03.901572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1407/2784] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:07:03.904894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:03.952708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e8683-8320-41d3-83ad-874e4d36eb87 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:03.958800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:03.960054 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:03.971265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:03.971627 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] PATH_INFO: `/v3/projects/75b40e3300ca47d99f4161af179ccb7e/users/cb2dfa0faf5044f5b32edd05b97d7719/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:03.978084 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b40e3300ca47d99f4161af179ccb7e, user_id=cb2dfa0faf5044f5b32edd05b97d7719, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:03.985387 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:04.011900 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2785] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:07:03 2022] PUT /identity/v3/projects/75b40e3300ca47d99f4161af179ccb7e/users/cb2dfa0faf5044f5b32edd05b97d7719/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:07:04.024296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fb7c5-ac57-4e44-b4ef-a73f23379e6c None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:04.024438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fb7c5-ac57-4e44-b4ef-a73f23379e6c None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:04.024631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fb7c5-ac57-4e44-b4ef-a73f23379e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:04.095487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-743fb7c5-ac57-4e44-b4ef-a73f23379e6c None None] MFA Rules not processed for user `cb2dfa0faf5044f5b32edd05b97d7719`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:04.096907 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-743fb7c5-ac57-4e44-b4ef-a73f23379e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:04.300948 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1408/2786] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:04.312236 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:04.314290 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:04.431642 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e81f3-af7b-4b27-8457-59ffcc8b05ba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:04.445247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:04.445934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:04.446226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:04.446509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:04.631428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:04.634987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:04.667770 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2787] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:04.823211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a365ea4-59d0-44ad-b73b-3aa4267adf43 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:04.824111 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a365ea4-59d0-44ad-b73b-3aa4267adf43 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:04.825215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a365ea4-59d0-44ad-b73b-3aa4267adf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:04.871943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-8a365ea4-59d0-44ad-b73b-3aa4267adf43 None None] MFA Rules not processed for user `70ba1847a4044ae6a338cdb4c80af5df`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:04.874233 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8a365ea4-59d0-44ad-b73b-3aa4267adf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:05.009384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1409/2788] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:05.018616 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:05.020923 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:05.117462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:05.182553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24bd78d-a85e-48e6-bc87-2abc2d7c2a00 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:05.193287 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:05.194112 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:05.194485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:05.194843 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:05.205030 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:05.207509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:05.218973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2789] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:07.006060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e44f3f-3a74-41f7-ade3-f8d9e8320895 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:07.006585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e44f3f-3a74-41f7-ade3-f8d9e8320895 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:07.006683 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e44f3f-3a74-41f7-ade3-f8d9e8320895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:07.041251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-37e44f3f-3a74-41f7-ade3-f8d9e8320895 None None] MFA Rules not processed for user `70ba1847a4044ae6a338cdb4c80af5df`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:07.042485 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-37e44f3f-3a74-41f7-ade3-f8d9e8320895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:07.056982 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1410/2790] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:07.540721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:07.696833 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edf340-0f2c-4d03-af26-e29acdaffaf4 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:07.704006 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:07.705545 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:07.706018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:07.706485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] PATH_INFO: `/v3/users/e6426ed3bbac4deba9b202d131c74f05` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:07.708676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6426ed3bbac4deba9b202d131c74f05)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:07.712099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:07.844506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6426ed3bbac4deba9b202d131c74f05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:07.845195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6426ed3bbac4deba9b202d131c74f05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:07.889192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6426ed3bbac4deba9b202d131c74f05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:07.901118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6426ed3bbac4deba9b202d131c74f05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:07.934542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6426ed3bbac4deba9b202d131c74f05'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:07.954841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2791] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:07:07 2022] DELETE /identity/v3/users/e6426ed3bbac4deba9b202d131c74f05 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:07.959487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:07.959855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:07.960229 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:07.993355 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:07.997811 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:08.081338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=100 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:08.131615 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1411/2792] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:08.142078 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:08.143910 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:08.279804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:08.329917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa47cb0-69be-4c9e-833b-8ae7da0494e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:08.337371 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05434e6d-8ef4-4589-8124-5850aa9af921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:08.339148 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05434e6d-8ef4-4589-8124-5850aa9af921 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:08.339449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05434e6d-8ef4-4589-8124-5850aa9af921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:08.339737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05434e6d-8ef4-4589-8124-5850aa9af921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:08.357840 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05434e6d-8ef4-4589-8124-5850aa9af921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:08.360948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05434e6d-8ef4-4589-8124-5850aa9af921 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:08.391462 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2793] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:08.832496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:08.838043 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:08.866196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ab719-8002-45a4-98a5-3a848378f124 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:08.871831 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:08.873400 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:08.874093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:08.874564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] PATH_INFO: `/v3/projects/d509204b7b144985be48fbfc93d14297` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:08.875993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d509204b7b144985be48fbfc93d14297)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:08.878451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:08.937793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:08.938384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:08.938384 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:08.960377 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] MFA Rules not processed for user `03318d8ab40e4aa5b0db0e893c79ec61`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:08.962081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:08.967284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd509204b7b144985be48fbfc93d14297', 'request_id': 'req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:08.978186 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd509204b7b144985be48fbfc93d14297', 'request_id': 'req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:08.986163 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd509204b7b144985be48fbfc93d14297', 'request_id': 'req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:08.999669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] The token cache is being invalidate because project d509204b7b144985be48fbfc93d14297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:07:08.999970 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.001155 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.004626 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1412/2794] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:07:08 2022] DELETE /identity/v3/projects/d509204b7b144985be48fbfc93d14297 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:09.006702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:09.008053 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:09.033227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2795] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:09.099751 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5b9e2-3162-4fc7-bb56-92c2c9defb15 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:09.109576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:09.110749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:09.111268 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:09.111652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] PATH_INFO: `/v3/users/cb2dfa0faf5044f5b32edd05b97d7719` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:09.113058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2dfa0faf5044f5b32edd05b97d7719)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:09.115106 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:09.216922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2dfa0faf5044f5b32edd05b97d7719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.226993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2dfa0faf5044f5b32edd05b97d7719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.227957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2dfa0faf5044f5b32edd05b97d7719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.237826 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:09.263897 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2dfa0faf5044f5b32edd05b97d7719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.289349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f1e798e1-4669-48ad-adee-dc0a8558a10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2dfa0faf5044f5b32edd05b97d7719'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.310312 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1413/2796] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:07:09 2022] DELETE /identity/v3/users/cb2dfa0faf5044f5b32edd05b97d7719 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:07:09.680238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:09.694046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b10f0-59f5-4f13-b240-7fd211f8562f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:09.781141 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:09.782319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:09.782608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:09.782891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] PATH_INFO: `/v3/projects/75b40e3300ca47d99f4161af179ccb7e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:09.784228 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b40e3300ca47d99f4161af179ccb7e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:09.786432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:09.842248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b40e3300ca47d99f4161af179ccb7e', 'request_id': 'req-fbcb192f-f8df-4f17-bd10-65bb0550cfee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.848283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b40e3300ca47d99f4161af179ccb7e', 'request_id': 'req-fbcb192f-f8df-4f17-bd10-65bb0550cfee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.857674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b40e3300ca47d99f4161af179ccb7e', 'request_id': 'req-fbcb192f-f8df-4f17-bd10-65bb0550cfee'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.877246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] The token cache is being invalidate because project 75b40e3300ca47d99f4161af179ccb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:07:09.877636 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.878541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fbcb192f-f8df-4f17-bd10-65bb0550cfee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:09.881801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2797] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:07:09 2022] DELETE /identity/v3/projects/75b40e3300ca47d99f4161af179ccb7e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:09.887090 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:09.887740 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:09.888032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:09.913581 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:09.915199 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:09.988575 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1414/2798] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:09.996633 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:09.997194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:09.997194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.016660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:10.018102 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:10.027752 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2799] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:10.031911 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.042037 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d73fe-b766-424a-9dcd-6ceb70754d3a None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.047432 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.048104 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.048564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.048850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.049777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:07:10.050265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.051968 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.065147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1415/2800] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:07:10.070361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.081987 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31137a17-73c6-4d17-ac93-9bc7550d7942 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.087020 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.088234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.088234 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.088505 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.090275 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.091960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.096931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:10.128069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2801] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:07:10 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:07:10.132696 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.145215 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d86594-65d4-44b4-8f8e-e200bb891b5f admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.233756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.234754 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.235146 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.235531 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.236721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.238918 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.308250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1416/2802] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:07:10 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:07:10.312600 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.331052 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac506-e64f-4c64-87ef-818acd95deea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.344475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.345358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.345744 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.346730 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.347580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.349328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.383325 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2803] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:07:10.384011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.398750 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ed4a2-e5a8-4b73-b4b4-b34c1a852e95 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.414843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0343cd-e258-4e93-9478-bf677d55caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.415852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0343cd-e258-4e93-9478-bf677d55caa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.416404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0343cd-e258-4e93-9478-bf677d55caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.417112 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0343cd-e258-4e93-9478-bf677d55caa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.418100 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0343cd-e258-4e93-9478-bf677d55caa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.419917 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0343cd-e258-4e93-9478-bf677d55caa4 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.431475 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1417/2804] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:07:10.435620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.458766 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b706db-c8e1-46c7-8835-ed472a770d31 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.467562 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.468785 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.468881 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.471646 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] PATH_INFO: `/v3/projects/7a096fd4ee214a9da5e2c9ed054f2092/users/fb71577800de46e78691e5e46d289014/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.512259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a096fd4ee214a9da5e2c9ed054f2092, user_id=fb71577800de46e78691e5e46d289014, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.518373 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.553058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2805] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:07:10 2022] PUT /identity/v3/projects/7a096fd4ee214a9da5e2c9ed054f2092/users/fb71577800de46e78691e5e46d289014/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:10.554627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eb5c7-ef2e-46d3-8cda-10c0da4b1553 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.555101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eb5c7-ef2e-46d3-8cda-10c0da4b1553 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.555379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0eb5c7-ef2e-46d3-8cda-10c0da4b1553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.603943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2d0eb5c7-ef2e-46d3-8cda-10c0da4b1553 None None] MFA Rules not processed for user `fb71577800de46e78691e5e46d289014`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:10.605196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2d0eb5c7-ef2e-46d3-8cda-10c0da4b1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:10.648031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:10.688889 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1418/2806] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:07:10.700966 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:10.703437 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:10.786406 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab3f53-bf73-4288-afc1-f372eff97756 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:10.791021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:10.791691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:10.791983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:10.792331 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:10.808227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:10.811150 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:10.834780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2807] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:11.863769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:11.864313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:11.864701 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:11.884613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:11.885974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:11.969338 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1419/2808] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Jul 19 13:07:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:11.981869 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:11.983316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:12.062655 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e53be-1ed4-471d-839c-1b274b201074 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:12.067931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:12.068759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:12.069178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:12.069558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:12.079074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:12.080827 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:12.089712 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2809] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 13:07:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 13:07:21.957461 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=36 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:22.086104 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=78,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:39.705515 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:39.707725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:39.781839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae9b5-86cd-40aa-9cdb-3dd8f0c13e99 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:39.785490 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:39.786142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:39.786425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:39.786702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:39.788492 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:39.798843 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:39.800332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:39.808553 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1420/2810] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:40.876887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:40.878329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:40.890043 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f216bc-1b26-4564-ab81-16730ecffcb9 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:40.893873 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:40.894513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:40.894801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:40.895080 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:40.896801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:40.958122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:40.959543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee678aac-c383-491e-9df4-adc527778241 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:40.969390 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:46.494922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:46.556950 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a383e-b8d4-46ad-bdb0-105aac9c38c8 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:46.561580 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:46.562495 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:46.562765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:46.563030 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] PATH_INFO: `/v3/users/fb71577800de46e78691e5e46d289014` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:46.564275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb71577800de46e78691e5e46d289014)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:46.566219 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:46.612269 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb71577800de46e78691e5e46d289014'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:46.619644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb71577800de46e78691e5e46d289014'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:46.620281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb71577800de46e78691e5e46d289014'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:46.632614 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb71577800de46e78691e5e46d289014'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:46.640007 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb71577800de46e78691e5e46d289014'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:46.646154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1421/2812] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:07:46 2022] DELETE /identity/v3/users/fb71577800de46e78691e5e46d289014 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:46.650796 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:46.651274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:46.651393 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:46.667973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:07:46.669194 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:46.726795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2813] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:07:46.736307 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:46.738021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:46.752500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:46.799241 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b19a1-95f5-4116-b2aa-7b86b2a9b99a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:46.802855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:46.803503 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:46.803789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:46.804074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:46.813546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:46.814990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:46.824378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1422/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:07:47.092780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:07:47.094229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:07:47.106187 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66680c6d-8a16-4a3d-b5da-3cb640e65db4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:07:47.111564 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:07:47.112558 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:07:47.112845 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:07:47.113867 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] PATH_INFO: `/v3/projects/7a096fd4ee214a9da5e2c9ed054f2092` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:07:47.115451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a096fd4ee214a9da5e2c9ed054f2092)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:07:47.117353 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:07:47.163704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a096fd4ee214a9da5e2c9ed054f2092', 'request_id': 'req-13a30056-1a9b-4621-8a7c-d45b518183f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:47.169191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a096fd4ee214a9da5e2c9ed054f2092', 'request_id': 'req-13a30056-1a9b-4621-8a7c-d45b518183f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:47.181165 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a096fd4ee214a9da5e2c9ed054f2092', 'request_id': 'req-13a30056-1a9b-4621-8a7c-d45b518183f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:47.192836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] The token cache is being invalidate because project 7a096fd4ee214a9da5e2c9ed054f2092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:07:47.193123 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:47.193764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:07:47.195289 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2815] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:07:47 2022] DELETE /identity/v3/projects/7a096fd4ee214a9da5e2c9ed054f2092 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:07:56.820120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=21 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:07:57.189865 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:08:10.300085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:08:10.308621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548dba8-48ed-404b-8636-e912044fc9d2 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:08:10.357671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:08:10.358296 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:08:10.358578 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:08:10.358852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:08:10.360653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:08:10.428224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:08:10.429657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:08:10.441305 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1423/2816] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:08:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:08:11.631894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:08:11.633333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:08:11.645817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a30056-1a9b-4621-8a7c-d45b518183f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:08:11.649409 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:08:11.650066 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:08:11.650358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:08:11.650641 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:08:11.652402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:08:11.666004 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:08:11.667430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:08:11.679097 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2817] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:08:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:08:20.436239 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=54 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:08:21.674144 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:08:54.496549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:08:54.506558 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:08:54.536224 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5363d2e-d916-40ce-9064-016356c52928 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:08:54.542047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:08:54.543016 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:08:54.543507 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:08:54.544127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:08:54.546302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:08:54.559514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:08:54.561402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978fdfd-5da7-424b-8a16-3d28358f793f service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:08:54.575800 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1424/2818] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:08:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:09:00.540219 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:00.541917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:00.556368 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b40ee-9fc3-4cd4-a5fb-928fe9e78de5 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:00.563405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:00.564046 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:00.564365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:00.564645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:00.566551 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:00.579691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:00.581316 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:00.592230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2819] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:01.900654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:01.901093 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:01.901342 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:01.919864 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] MFA Rules not processed for user `71391e0896644c0a82a06b0905b9df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:01.921096 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:01.977723 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1425/2820] 10.222.0.13 () {58 vars in 967 bytes} [Tue Jul 19 13:09:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:02.018981 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:02.020424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:02.088841 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c34c2e-1b81-48b6-83b8-46d7257b5e09 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:02.093738 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2205868e3b04113b500a67eb814e58a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:02.094374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:02.094661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:02.094939 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:02.102847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:02.104515 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:02.113100 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2821] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Jul 19 13:09:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 19 13:09:09.056446 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:09.058069 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:09.132328 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06615d-3657-43b4-b862-2db3400d3dd8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:09.136858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:09.137726 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:09.138011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:09.138291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] PATH_INFO: `/v3/users/03318d8ab40e4aa5b0db0e893c79ec61` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:09.139487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03318d8ab40e4aa5b0db0e893c79ec61)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:09.141996 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:09.192513 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03318d8ab40e4aa5b0db0e893c79ec61'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.200998 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03318d8ab40e4aa5b0db0e893c79ec61'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.201678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03318d8ab40e4aa5b0db0e893c79ec61'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.215538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03318d8ab40e4aa5b0db0e893c79ec61'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.226542 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03318d8ab40e4aa5b0db0e893c79ec61'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.232705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1426/2822] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:09:09 2022] DELETE /identity/v3/users/03318d8ab40e4aa5b0db0e893c79ec61 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:09:09.441647 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:09.460615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6733f0-7028-491d-97bc-a0ea8087abf6 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:09.545797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:09.547519 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:09.547834 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:09.548088 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] PATH_INFO: `/v3/projects/655b99de30324b49b9d33a3f3f4051a8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:09.549346 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=655b99de30324b49b9d33a3f3f4051a8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:09.551398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:09.602813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655b99de30324b49b9d33a3f3f4051a8', 'request_id': 'req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.610074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655b99de30324b49b9d33a3f3f4051a8', 'request_id': 'req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.619804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655b99de30324b49b9d33a3f3f4051a8', 'request_id': 'req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.631595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] The token cache is being invalidate because project 655b99de30324b49b9d33a3f3f4051a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:09:09.631887 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.632553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.634081 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2823] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:09:09 2022] DELETE /identity/v3/projects/655b99de30324b49b9d33a3f3f4051a8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:09:09.638013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:09.639445 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:09.652819 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099370ae-c1ef-49be-81ea-2a15a58cd829 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:09.660868 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:09.712393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:09.713272 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:09.713554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:09.713827 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] PATH_INFO: `/v3/users/99b4911b6fcb47fe82fb4eee4758c607` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:09.715023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b4911b6fcb47fe82fb4eee4758c607)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:09.717085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:09.766561 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b4911b6fcb47fe82fb4eee4758c607'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.774838 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b4911b6fcb47fe82fb4eee4758c607'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.775685 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b4911b6fcb47fe82fb4eee4758c607'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.789284 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b4911b6fcb47fe82fb4eee4758c607'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.796358 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c6e5d11c-99b0-437a-b0fd-983073f3c384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b4911b6fcb47fe82fb4eee4758c607'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:09.803210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1427/2824] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:09:09 2022] DELETE /identity/v3/users/99b4911b6fcb47fe82fb4eee4758c607 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:09:10.005280 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.006970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:10.020426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650590a-2f7e-44d0-b730-d2fb1e6d6e68 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.024676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:10.089426 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.090293 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.090585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.090868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] PATH_INFO: `/v3/projects/57f6377da3f7422282b53e42aba2be15` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.092059 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f6377da3f7422282b53e42aba2be15)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.094503 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.139416 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f6377da3f7422282b53e42aba2be15', 'request_id': 'req-4d5479c3-b69a-4ae3-821e-e402e1bed286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:10.147897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f6377da3f7422282b53e42aba2be15', 'request_id': 'req-4d5479c3-b69a-4ae3-821e-e402e1bed286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:10.161192 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f6377da3f7422282b53e42aba2be15', 'request_id': 'req-4d5479c3-b69a-4ae3-821e-e402e1bed286'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:10.174135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] The token cache is being invalidate because project 57f6377da3f7422282b53e42aba2be15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:09:10.174452 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:10.175108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4d5479c3-b69a-4ae3-821e-e402e1bed286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:09:10.176680 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2825] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:09:10 2022] DELETE /identity/v3/projects/57f6377da3f7422282b53e42aba2be15 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:09:10.181730 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.182208 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.182291 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.200449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:10.201665 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:10.262189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1428/2826] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:10.271485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.271774 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.272061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.291099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:10.292335 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:10.300870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2827] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:10.304986 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.314630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5658a-2990-4844-a7ff-682177a9f029 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.319316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.319839 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.321052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.321052 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.321225 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:09:10.321434 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.323023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.335404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1429/2828] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:09:10.339500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.353322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56bacf-bb3b-4a86-a95f-8a59c01ec3a0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.357944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.358596 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.358912 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.359196 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.359891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.361826 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.388220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2829] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:09:10 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:09:10.392990 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.403083 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15846b0-3340-43e7-8edb-69fe93d0481a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.417791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:10.467836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.469164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.470564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.470564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.470564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.472401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.492010 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1430/2830] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:09:10 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:09:10.496132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.508209 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e327671-f615-41ef-898d-6d6412cefe66 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.513039 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.513690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.513975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.514252 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.514973 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.517175 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.528058 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2831] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:09:10.532138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.542131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b39d2-6a4a-41f1-b8f5-d786268a806c admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.546728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5be7c-da04-48c0-bd7d-2d92a317aab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.547375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5be7c-da04-48c0-bd7d-2d92a317aab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.547667 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5be7c-da04-48c0-bd7d-2d92a317aab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.547946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5be7c-da04-48c0-bd7d-2d92a317aab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.548732 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5be7c-da04-48c0-bd7d-2d92a317aab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.550339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5be7c-da04-48c0-bd7d-2d92a317aab5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.560126 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1431/2832] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:09:10.564476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.575445 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad58c68-a26d-4c19-9c70-1cea239e0784 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.580628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.581514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.581800 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.582053 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] PATH_INFO: `/v3/projects/3de65ab2754b49d69c41eb4d2870648e/users/97565560b942439c9c4c09bf3097b8c8/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.596610 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de65ab2754b49d69c41eb4d2870648e, user_id=97565560b942439c9c4c09bf3097b8c8, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.599274 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.621212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2833] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:09:10 2022] PUT /identity/v3/projects/3de65ab2754b49d69c41eb4d2870648e/users/97565560b942439c9c4c09bf3097b8c8/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:09:10.626772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795fc56-eff5-4581-8adf-5460d2dce2bc None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.627132 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795fc56-eff5-4581-8adf-5460d2dce2bc None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.627425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795fc56-eff5-4581-8adf-5460d2dce2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.646194 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-6795fc56-eff5-4581-8adf-5460d2dce2bc None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:10.647417 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-6795fc56-eff5-4581-8adf-5460d2dce2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:10.713275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1432/2834] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:10.723118 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:10.725329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:10.761405 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:10.787488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463e190-2142-491c-bb6d-bc6f372f6de5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:10.791199 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:10.792107 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:10.792319 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:10.792612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:10.802817 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:10.804404 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:10.815000 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2835] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:14.545350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:14.545807 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:14.545949 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:14.574966 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] MFA Rules not processed for user `97565560b942439c9c4c09bf3097b8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:14.576210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:14.650769 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1433/2836] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:14.656297 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:14.666622 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79765956-5c99-4b5e-8807-ce7c57389b7c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:14.673615 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:14.674284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:14.674575 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:14.674855 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:14.675560 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:14.677521 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:14.707542 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2837] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:09:14 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:09:14.711702 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:14.723983 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e20ec3-b3a0-48d1-aa56-805e5bcbf9c8 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:14.748763 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:14.787238 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:14.787922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:14.788255 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:14.788514 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:14.789221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:14.791152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:14.816246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1434/2838] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:09:14 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:09:14.817685 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:14.829007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4616-d65b-431c-8925-f4a4af0846b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:14.833899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:14.834601 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:14.834891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:14.835166 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:14.835898 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:14.837643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:14.849890 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2839] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:09:14.854128 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:14.864627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3167-fdf5-4e8c-a28b-895bc21b1f3b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:14.869717 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:14.870694 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:14.871059 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:14.871412 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] PATH_INFO: `/v3/projects/545016fd05654847998ed6f7ba9339fa/users/05886636d6ab456db9297e082dbf0920/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:14.887742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=545016fd05654847998ed6f7ba9339fa, user_id=05886636d6ab456db9297e082dbf0920, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:14.891027 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:14.906636 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1435/2840] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:09:14 2022] PUT /identity/v3/projects/545016fd05654847998ed6f7ba9339fa/users/05886636d6ab456db9297e082dbf0920/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:09:14.910605 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:14.921931 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe7543-b48d-46fd-9d42-2479f9154dd7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:14.982226 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b029990-da3c-4d4c-8067-178b309ff0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:14.982905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029990-da3c-4d4c-8067-178b309ff0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:14.983207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029990-da3c-4d4c-8067-178b309ff0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:14.983486 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029990-da3c-4d4c-8067-178b309ff0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:14.984365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029990-da3c-4d4c-8067-178b309ff0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:14.985969 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029990-da3c-4d4c-8067-178b309ff0ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:14.996322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2841] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:09:15.000974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:15.011378 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdeffa7-50a8-47a4-90f1-9b141e45ded3 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:15.016776 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:15.017654 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:15.017944 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:15.018227 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] PATH_INFO: `/v3/projects/545016fd05654847998ed6f7ba9339fa/users/05886636d6ab456db9297e082dbf0920/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:15.020491 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=545016fd05654847998ed6f7ba9339fa, user_id=05886636d6ab456db9297e082dbf0920, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:15.023125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:15.033348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1436/2842] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:09:14 2022] PUT /identity/v3/projects/545016fd05654847998ed6f7ba9339fa/users/05886636d6ab456db9297e082dbf0920/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:09:19.843016 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2275e-eb9a-40ab-8fcb-072ae8afbd31 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:19.844160 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2275e-eb9a-40ab-8fcb-072ae8afbd31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:19.844488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2275e-eb9a-40ab-8fcb-072ae8afbd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:19.879794 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-06e2275e-eb9a-40ab-8fcb-072ae8afbd31 None None] MFA Rules not processed for user `05886636d6ab456db9297e082dbf0920`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:19.881223 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06e2275e-eb9a-40ab-8fcb-072ae8afbd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:19.958278 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2843] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:19.972879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:19.974768 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:20.054463 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6e587-9610-4662-8eff-ae1e0ac28d4b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:20.059955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:20.060718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:20.061001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:20.061276 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:20.121621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:20.123365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:20.133460 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1437/2844] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:21.129229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:21.129692 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:21.129801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:21.270793 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] MFA Rules not processed for user `97565560b942439c9c4c09bf3097b8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:21.271999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:21.279311 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2845] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:30.129300 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:31.142776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:35.663154 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:35.664638 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:35.730236 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1dde5-4138-4a5d-a66d-c8d53bb549de service nova] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:35.733848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:35.735045 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:35.735332 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:35.735612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:35.737383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:35.746906 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:35.748370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:35.757001 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1438/2846] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:36.186447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:36.187952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:36.301178 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb613824-3dc7-4b30-9006-a1487be877c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:36.307069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:36.307847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:36.308138 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:36.308451 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:36.310652 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:36.322157 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:36.323718 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:36.334352 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2847] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:45.220133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:45.232747 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4df13-403c-435e-9571-253215226188 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:45.238544 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:45.239188 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:45.239478 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:45.239759 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:45.241538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:45.253360 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:45.255067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8573eef-c578-45cc-a77f-70233092681e service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:45.265402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1439/2848] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:46.329820 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=50 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:09:49.537284 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:49.538819 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:49.555698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72d073-6d70-401d-bb01-81e2f3a4b7b8 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:49.561847 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99d07f11438d4e069eab093a8b99eeab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:49.562509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:49.562801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:49.563108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:49.565085 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:49.578069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:49.579676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:49.590944 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2849] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:09:51.086256 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:51.086808 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:51.086958 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:51.106691 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] MFA Rules not processed for user `70ba1847a4044ae6a338cdb4c80af5df`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:09:51.107880 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:09:51.167029 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1440/2850] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:09:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:09:51.841691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:09:51.853161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ff6fd-4fca-4a39-883b-23415b7da319 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:09:51.858036 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:09:51.864916 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:09:51.865727 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:09:51.866212 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:09:51.878355 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:09:51.879777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:09:51.890454 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2851] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:10:01.159346 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=61 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:10:01.885626 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=12 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:11.676632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:11.678646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:11.760623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c054ef9-a4bd-4dec-a64e-7f425a547e89 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:11.765209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:11.766086 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:11.766374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:11.766653 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] PATH_INFO: `/v3/users/70ba1847a4044ae6a338cdb4c80af5df` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:11.767883 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70ba1847a4044ae6a338cdb4c80af5df)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:11.770141 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:11.817260 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70ba1847a4044ae6a338cdb4c80af5df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:11.826409 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70ba1847a4044ae6a338cdb4c80af5df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:11.827023 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70ba1847a4044ae6a338cdb4c80af5df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:11.839591 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70ba1847a4044ae6a338cdb4c80af5df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:11.847063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-0dec8744-5e9b-4cf2-b2b7-8f79330646e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70ba1847a4044ae6a338cdb4c80af5df'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:11.852857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1441/2852] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:11 2022] DELETE /identity/v3/users/70ba1847a4044ae6a338cdb4c80af5df => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:12.022458 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.037385 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668b679-0328-4c51-be41-66e4de5362d2 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.094137 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.094765 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.095076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.095382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] PATH_INFO: `/v3/projects/491e79a36d5b4f8cb3952be89ae6c03d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.097328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] RBAC: Authorizing `identity:delete_project(project_id=491e79a36d5b4f8cb3952be89ae6c03d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.099526 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.145241 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '491e79a36d5b4f8cb3952be89ae6c03d', 'request_id': 'req-d699d1ff-0081-4b3e-a682-f0823ac0f26c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:12.150693 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '491e79a36d5b4f8cb3952be89ae6c03d', 'request_id': 'req-d699d1ff-0081-4b3e-a682-f0823ac0f26c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:12.157585 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '491e79a36d5b4f8cb3952be89ae6c03d', 'request_id': 'req-d699d1ff-0081-4b3e-a682-f0823ac0f26c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:12.168823 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] The token cache is being invalidate because project 491e79a36d5b4f8cb3952be89ae6c03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:12.169115 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:12.169739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d699d1ff-0081-4b3e-a682-f0823ac0f26c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:12.171240 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2853] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:12 2022] DELETE /identity/v3/projects/491e79a36d5b4f8cb3952be89ae6c03d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:12.176775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.177067 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.177350 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.195169 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:12.197182 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:12.389174 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1442/2854] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:12.397868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.398374 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.398664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.417152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:12.418351 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:12.426531 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2855] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:12.430369 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.440021 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ce15f-1c85-4fd8-bdd9-cea1280cc198 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.444721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.445381 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.445697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.445976 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.446742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:11:12.447050 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.448690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.450209 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:12.463265 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1443/2856] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:11:12.467472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.477247 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d906-5c94-4cc6-80e7-9940a541dbb1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.481677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.482314 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.482595 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.482866 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.483541 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.485449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.510022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2857] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/projects => generated 403 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:12.514176 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.526899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c72ef4-664f-43df-a51f-e2711c53a039 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.599330 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.600345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.600777 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.601189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.602339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.604482 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.628703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1444/2858] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:11:12.632924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.644358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe1f8-4d02-43d3-ba98-08860773bf7c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.648925 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.649573 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.649857 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.650132 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.650849 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.652479 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.662635 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2859] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:11:12.666435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.676251 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088314d0-1661-4ecf-80ad-87f50540c009 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.680699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f5b3c-5b82-48ad-9b89-7c7ae9bb47f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.681348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5b3c-5b82-48ad-9b89-7c7ae9bb47f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.681631 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5b3c-5b82-48ad-9b89-7c7ae9bb47f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.681908 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f5b3c-5b82-48ad-9b89-7c7ae9bb47f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.682627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5b3c-5b82-48ad-9b89-7c7ae9bb47f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.684254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f5b3c-5b82-48ad-9b89-7c7ae9bb47f8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.694787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1445/2860] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:12.698737 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.709213 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d54686-4365-4dd1-8187-3c4dcbaba75f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.713644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.714506 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.714790 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.715065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] PATH_INFO: `/v3/projects/9aadfbb4c70e43edacc936a1dd936ad2/users/e627b82e910d4b63b6e273ed5f0fe2cc/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.729167 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aadfbb4c70e43edacc936a1dd936ad2, user_id=e627b82e910d4b63b6e273ed5f0fe2cc, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.731777 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.744170 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2861] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:12 2022] PUT /identity/v3/projects/9aadfbb4c70e43edacc936a1dd936ad2/users/e627b82e910d4b63b6e273ed5f0fe2cc/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:12.755222 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.755512 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.755793 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.781672 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] MFA Rules not processed for user `e627b82e910d4b63b6e273ed5f0fe2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:12.782851 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:12.845028 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1446/2862] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:12.849739 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.874381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:12.914259 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5edca-31ea-4d65-9ab2-4af38b317b08 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:12.925320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:12.925983 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:12.926318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:12.926548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:12.929532 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:12.931424 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:12.961664 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2863] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:12.965887 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:12.976438 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee7afe-89a6-4a74-829f-fcf2f93747b7 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.031066 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.031725 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.032009 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.032316 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.033000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.035102 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.042435 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:13.055166 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1447/2864] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:11:12 2022] POST /identity/v3/users => generated 309 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:11:13.059131 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.070090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d26cba-4fb4-4e12-9c01-762eb3f70456 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.074628 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.075283 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.075570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.075851 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.076618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.078231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.088517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2865] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:13.092401 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.102181 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020cab8-cac9-4503-a424-712394067f14 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.106693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.107564 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.107846 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.108121 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] PATH_INFO: `/v3/projects/dc196c85fb994a6ca8082d376c5e5dee/users/db48d2d6dcf84662bda610fb96505e66/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.123047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc196c85fb994a6ca8082d376c5e5dee, user_id=db48d2d6dcf84662bda610fb96505e66, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.125791 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.139860 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1448/2866] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:13 2022] PUT /identity/v3/projects/dc196c85fb994a6ca8082d376c5e5dee/users/db48d2d6dcf84662bda610fb96505e66/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:11:13.143899 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.153676 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1de439-afd6-435f-a81b-106c142e6cc0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.207870 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500c4d9-e508-4da1-8106-d3de077c9368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.208806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500c4d9-e508-4da1-8106-d3de077c9368 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.209091 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500c4d9-e508-4da1-8106-d3de077c9368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.209360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500c4d9-e508-4da1-8106-d3de077c9368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.210070 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500c4d9-e508-4da1-8106-d3de077c9368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.211640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500c4d9-e508-4da1-8106-d3de077c9368 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.221343 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2867] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:13.225483 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.236213 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fd356-9b5c-44fd-a74a-e09765444647 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.241898 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.242782 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.243068 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.243348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] PATH_INFO: `/v3/projects/dc196c85fb994a6ca8082d376c5e5dee/users/db48d2d6dcf84662bda610fb96505e66/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.245374 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc196c85fb994a6ca8082d376c5e5dee, user_id=db48d2d6dcf84662bda610fb96505e66, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.248047 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.259393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1449/2868] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:13 2022] PUT /identity/v3/projects/dc196c85fb994a6ca8082d376c5e5dee/users/db48d2d6dcf84662bda610fb96505e66/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:11:13.270305 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.270593 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.270897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.296948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] MFA Rules not processed for user `db48d2d6dcf84662bda610fb96505e66`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:13.298127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:13.366257 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2869] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:13.376687 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.378415 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:13.454281 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448015e-c2fd-421c-a8da-3318ff9c0c3e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.457857 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7cf77-9871-47c1-b417-4c95532bc0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.458549 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7cf77-9871-47c1-b417-4c95532bc0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.458780 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7cf77-9871-47c1-b417-4c95532bc0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.459055 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7cf77-9871-47c1-b417-4c95532bc0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.467743 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7cf77-9871-47c1-b417-4c95532bc0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.469189 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7cf77-9871-47c1-b417-4c95532bc0d9 service cinder] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.478856 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1450/2870] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:13.608415 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.628093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:13.671836 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545e13e-3c83-445a-9926-383f90ee03da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.677122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.678012 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.678279 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.678586 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] PATH_INFO: `/v3/users/e627b82e910d4b63b6e273ed5f0fe2cc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.679758 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e627b82e910d4b63b6e273ed5f0fe2cc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.681703 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.727846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e627b82e910d4b63b6e273ed5f0fe2cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:13.728517 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e627b82e910d4b63b6e273ed5f0fe2cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:13.735863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e627b82e910d4b63b6e273ed5f0fe2cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:13.742940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e627b82e910d4b63b6e273ed5f0fe2cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:13.754555 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e627b82e910d4b63b6e273ed5f0fe2cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:13.760514 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2871] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:13 2022] DELETE /identity/v3/users/e627b82e910d4b63b6e273ed5f0fe2cc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:13.765117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.765560 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.765677 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.783453 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:13.785157 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:13.844596 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1451/2872] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:13.857461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:13.859444 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:13.921439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd664f3-1fb2-4e00-abb3-c2c591aad2f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:13.925128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:13.925775 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:13.926068 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:13.926350 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:13.935216 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:13.936681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:13.945919 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2873] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:14.213506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:14.214922 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:14.226579 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88947d-22b5-4ccc-a6c0-69ae0f50e7cd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:14.231114 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:14.231972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:14.232282 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:14.232565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] PATH_INFO: `/v3/projects/9aadfbb4c70e43edacc936a1dd936ad2` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:14.233745 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aadfbb4c70e43edacc936a1dd936ad2)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:14.235648 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:14.273621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:INSERT=4,keystone:SELECT=94,keystone:DELETE=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:14.284496 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aadfbb4c70e43edacc936a1dd936ad2', 'request_id': 'req-e4b8fdc5-b445-4181-a328-1199fd30f95b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.292589 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aadfbb4c70e43edacc936a1dd936ad2', 'request_id': 'req-e4b8fdc5-b445-4181-a328-1199fd30f95b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.299152 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aadfbb4c70e43edacc936a1dd936ad2', 'request_id': 'req-e4b8fdc5-b445-4181-a328-1199fd30f95b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.309476 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] The token cache is being invalidate because project 9aadfbb4c70e43edacc936a1dd936ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:14.309765 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.310405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.311923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1452/2874] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:14 2022] DELETE /identity/v3/projects/9aadfbb4c70e43edacc936a1dd936ad2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:14.316670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:14.326090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc170b9-fda2-4348-8b11-7991e61b4fcb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:14.363874 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:14.383905 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:14.384808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:14.385099 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:14.385383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] PATH_INFO: `/v3/users/db48d2d6dcf84662bda610fb96505e66` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:14.386569 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=db48d2d6dcf84662bda610fb96505e66)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:14.388513 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:14.435230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db48d2d6dcf84662bda610fb96505e66'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.435882 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db48d2d6dcf84662bda610fb96505e66'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.443708 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db48d2d6dcf84662bda610fb96505e66'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.450570 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db48d2d6dcf84662bda610fb96505e66'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.461935 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdc799ba-86fc-4891-9500-67f639477d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db48d2d6dcf84662bda610fb96505e66'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.467675 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2875] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:14 2022] DELETE /identity/v3/users/db48d2d6dcf84662bda610fb96505e66 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:14.733372 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:14.734836 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:14.757060 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8fdc5-b445-4181-a328-1199fd30f95b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:14.813470 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:14.814258 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:14.814543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:14.815375 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] PATH_INFO: `/v3/projects/dc196c85fb994a6ca8082d376c5e5dee` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:14.816595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc196c85fb994a6ca8082d376c5e5dee)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:14.818506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:14.863669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc196c85fb994a6ca8082d376c5e5dee', 'request_id': 'req-c38da7cb-4528-477a-b650-30b4135aad52'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.870637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc196c85fb994a6ca8082d376c5e5dee', 'request_id': 'req-c38da7cb-4528-477a-b650-30b4135aad52'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.876749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc196c85fb994a6ca8082d376c5e5dee', 'request_id': 'req-c38da7cb-4528-477a-b650-30b4135aad52'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.886828 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] The token cache is being invalidate because project dc196c85fb994a6ca8082d376c5e5dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:14.887117 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.887742 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-c38da7cb-4528-477a-b650-30b4135aad52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:14.889264 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1453/2876] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:14 2022] DELETE /identity/v3/projects/dc196c85fb994a6ca8082d376c5e5dee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:14.894509 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:14.894808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:14.895098 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:14.913207 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:14.914401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:14.973484 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2877] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:14.981206 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:14.981497 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:14.981804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.000451 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:15.001641 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:15.009488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1454/2878] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:15.013459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.023261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac45734-9ad2-4126-bf10-78fbbe9758d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.027822 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.028500 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.028792 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.029074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.029797 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:11:15.030106 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.031690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.043510 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2879] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:11:15.047458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.057145 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7747f5-af05-4d6b-9750-a48f0e60681f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.061849 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.062486 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.062803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.063081 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.063809 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.065863 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.090379 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1455/2880] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:11:15 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:15.094457 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.104940 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a328a-73cb-4e3a-949f-a58f008434bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.148265 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:15.162625 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.163333 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.163618 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.163894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.164620 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.166594 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.186550 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2881] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:11:15 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:11:15.190678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.200804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8d1f6-b4b6-4f3e-bb8f-6bf659cb69eb admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.205803 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.206455 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.206741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.207049 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.207772 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.209404 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.219506 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1456/2882] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 19 13:11:15.223698 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.233304 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c42a0-4918-41a2-b57a-3192ddd83e36 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.237825 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635ceb6-174d-42a5-8248-ced32b98a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.238464 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ceb6-174d-42a5-8248-ced32b98a0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.238748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ceb6-174d-42a5-8248-ced32b98a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.239022 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635ceb6-174d-42a5-8248-ced32b98a0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.239728 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635ceb6-174d-42a5-8248-ced32b98a0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.241358 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635ceb6-174d-42a5-8248-ced32b98a0c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.251673 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2883] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:15.255627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.265488 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffae21b-3af3-450c-9ef8-1839ac129be6 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.270934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.271804 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.272089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.272399 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] PATH_INFO: `/v3/projects/9299b6a34c7f4424aafa0687d291d068/users/cc6c8464fce64e3da08e56c15c086b9a/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.283262 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:15.288326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9299b6a34c7f4424aafa0687d291d068, user_id=cc6c8464fce64e3da08e56c15c086b9a, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.291469 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.304337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1457/2884] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:15 2022] PUT /identity/v3/projects/9299b6a34c7f4424aafa0687d291d068/users/cc6c8464fce64e3da08e56c15c086b9a/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:15.315808 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.316048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.316423 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.342191 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] MFA Rules not processed for user `cc6c8464fce64e3da08e56c15c086b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:15.343439 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:15.407264 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2885] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:15.412285 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.473031 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50485b-215c-4af0-b798-f1f659e830b1 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.477585 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.478235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.478518 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.478794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.479487 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.481405 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.505143 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1458/2886] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:11:15 2022] POST /identity/v3/projects => generated 363 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:15.509156 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.520135 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea64ed6-fc46-423f-aa66-ebe09c8320ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.574021 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.574607 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.574889 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.575164 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.575846 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.577805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.596459 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2887] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:11:15 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:11:15.600440 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.612150 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fcb8a-8c5b-44f0-8542-1e5d14cb9091 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.616771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.617424 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.617708 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.617985 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.618728 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.620347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.631196 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1459/2888] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:15.635348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.646105 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3af928-1a76-4e15-b0e6-e26108e32d0f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.650658 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.651522 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.651804 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.652079 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] PATH_INFO: `/v3/projects/a5b6bd0cbb404ada8874a964a837fcdc/users/22d913872c094462b1ab0130017280fe/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.666389 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b6bd0cbb404ada8874a964a837fcdc, user_id=22d913872c094462b1ab0130017280fe, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.669031 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.681428 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2889] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:15 2022] PUT /identity/v3/projects/a5b6bd0cbb404ada8874a964a837fcdc/users/22d913872c094462b1ab0130017280fe/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:15.685131 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.695718 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08093ff0-b80e-4ee3-9bcd-101eb2b76559 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.749651 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358c0ed-814a-4f6d-a46f-9251f6682ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.750313 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358c0ed-814a-4f6d-a46f-9251f6682ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.750598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358c0ed-814a-4f6d-a46f-9251f6682ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.750904 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358c0ed-814a-4f6d-a46f-9251f6682ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.751632 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358c0ed-814a-4f6d-a46f-9251f6682ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.753801 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358c0ed-814a-4f6d-a46f-9251f6682ce1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.763329 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1460/2890] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:15.767383 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.778193 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89074733-6a78-40ef-9c63-6dff85c1b117 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.782704 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.783567 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.783848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.784121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] PATH_INFO: `/v3/projects/a5b6bd0cbb404ada8874a964a837fcdc/users/22d913872c094462b1ab0130017280fe/roles/ac904f211a8f450fbf556c879296af4b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.786093 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b6bd0cbb404ada8874a964a837fcdc, user_id=22d913872c094462b1ab0130017280fe, role_id=ac904f211a8f450fbf556c879296af4b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.788764 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:15.799133 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2891] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:15 2022] PUT /identity/v3/projects/a5b6bd0cbb404ada8874a964a837fcdc/users/22d913872c094462b1ab0130017280fe/roles/ac904f211a8f450fbf556c879296af4b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:15.809125 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.809422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.809709 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.837964 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] MFA Rules not processed for user `22d913872c094462b1ab0130017280fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:15.838546 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:15.884289 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:15.906881 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1461/2892] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:15.917952 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:15.919661 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:15.930453 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:15.978907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cf85a-36b2-4f6f-a3c9-f0b3ab74f9eb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:15.982435 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b52e5-7254-4593-a86c-b880e9c194b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:15.983065 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b52e5-7254-4593-a86c-b880e9c194b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:15.983348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b52e5-7254-4593-a86c-b880e9c194b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:15.983621 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b52e5-7254-4593-a86c-b880e9c194b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:15.992868 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b52e5-7254-4593-a86c-b880e9c194b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:15.994286 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b52e5-7254-4593-a86c-b880e9c194b9 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:16.004354 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:16.393921 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:16.455779 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98acc9f-840a-43d2-8d5f-73854a84196f None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:16.460429 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:16.461310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:16.461595 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:16.461873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] PATH_INFO: `/v3/users/cc6c8464fce64e3da08e56c15c086b9a` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:16.463085 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6c8464fce64e3da08e56c15c086b9a)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:16.465034 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:16.511325 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6c8464fce64e3da08e56c15c086b9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:16.517915 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6c8464fce64e3da08e56c15c086b9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:16.518787 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6c8464fce64e3da08e56c15c086b9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:16.531127 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6c8464fce64e3da08e56c15c086b9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:16.538365 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6c8464fce64e3da08e56c15c086b9a'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:16.544164 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1462/2894] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:16 2022] DELETE /identity/v3/users/cc6c8464fce64e3da08e56c15c086b9a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:16.548776 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:16.549786 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:16.549948 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:16.566329 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:16.567543 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:16.625338 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2895] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:16.634994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:16.636421 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:16.692992 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720de963-15ba-4812-9ac3-a600fcf3cc3a admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:16.696630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:16.697266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:16.697555 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:16.697841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:16.707247 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:16.708721 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:16.718000 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1463/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:16.983366 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:16.984824 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:16.996979 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d7f6d-2ad8-4571-8927-f681e3b215f8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.001666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.002604 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.002963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.003313 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] PATH_INFO: `/v3/projects/9299b6a34c7f4424aafa0687d291d068` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.004548 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9299b6a34c7f4424aafa0687d291d068)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.006461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.053007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9299b6a34c7f4424aafa0687d291d068', 'request_id': 'req-0ce65170-df2b-4567-a93e-113a43c731ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.058318 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9299b6a34c7f4424aafa0687d291d068', 'request_id': 'req-0ce65170-df2b-4567-a93e-113a43c731ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.066852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9299b6a34c7f4424aafa0687d291d068', 'request_id': 'req-0ce65170-df2b-4567-a93e-113a43c731ef'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.077660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] The token cache is being invalidate because project 9299b6a34c7f4424aafa0687d291d068 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:17.077951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.078580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.080928 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2897] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:16 2022] DELETE /identity/v3/projects/9299b6a34c7f4424aafa0687d291d068 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:17.084656 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:17.094669 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a40a8-65b1-4530-a8b4-d9f1d25a7119 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.098473 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:17.153101 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.153923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.154202 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.154474 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] PATH_INFO: `/v3/users/22d913872c094462b1ab0130017280fe` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.155633 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d913872c094462b1ab0130017280fe)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.157584 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.200753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d913872c094462b1ab0130017280fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.208018 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d913872c094462b1ab0130017280fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.208657 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d913872c094462b1ab0130017280fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.220295 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d913872c094462b1ab0130017280fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.227064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-d8745a66-b774-49dd-af1d-2f65898c9095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d913872c094462b1ab0130017280fe'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.232824 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1464/2898] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:17 2022] DELETE /identity/v3/users/22d913872c094462b1ab0130017280fe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:17.500203 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:17.501720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:17.520152 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce65170-df2b-4567-a93e-113a43c731ef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.551204 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:17.579388 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.580438 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.580779 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.581122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] PATH_INFO: `/v3/projects/a5b6bd0cbb404ada8874a964a837fcdc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.582317 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b6bd0cbb404ada8874a964a837fcdc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.584224 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.630398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b6bd0cbb404ada8874a964a837fcdc', 'request_id': 'req-78d7f29a-af58-4502-9397-22c47d3bda7a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.636042 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b6bd0cbb404ada8874a964a837fcdc', 'request_id': 'req-78d7f29a-af58-4502-9397-22c47d3bda7a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.642938 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b6bd0cbb404ada8874a964a837fcdc', 'request_id': 'req-78d7f29a-af58-4502-9397-22c47d3bda7a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.653897 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] The token cache is being invalidate because project a5b6bd0cbb404ada8874a964a837fcdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:17.654180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.654795 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-78d7f29a-af58-4502-9397-22c47d3bda7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:17.656320 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2899] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:17 2022] DELETE /identity/v3/projects/a5b6bd0cbb404ada8874a964a837fcdc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:17.661955 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.662252 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.662538 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.680714 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:17.681841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:17.739572 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1465/2900] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:17.747381 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.747670 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.747951 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.766476 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:17.767682 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:17.775590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2901] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 19 13:11:17.779630 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:17.789200 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991f34-67e1-4e2b-96e9-4d3d3b40c7f6 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.793678 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.794324 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.794610 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.794886 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.795606 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:11:17.796033 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.797663 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.809613 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1466/2902] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:11:17.813544 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:17.823136 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292691c-5f13-4b4d-9a1b-d5189efd434d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.827587 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.828239 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.828523 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.828799 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.829472 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.831341 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.860884 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1437/2903] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:11:17 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:17.865697 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:17.875789 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86176a47-d138-4b55-b6e2-542bac2726a0 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.884413 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:17.932873 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.933462 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.933748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.934026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.934703 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.936664 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.954621 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1467/2904] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:11:17 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:17.958614 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:17.968679 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201fdadb-72be-44ec-a3d5-4f47bbde4d95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:17.973720 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:17.974432 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:17.974697 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:17.974975 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:17.975690 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:17.977342 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:17.988027 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1438/2905] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:17.991848 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:18.001705 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe65f2-bed4-405e-af28-91b17da25861 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:18.006210 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ad4c6-534b-4900-a959-28fc0899c2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:18.006853 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ad4c6-534b-4900-a959-28fc0899c2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.007142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ad4c6-534b-4900-a959-28fc0899c2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.007422 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ad4c6-534b-4900-a959-28fc0899c2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.008170 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ad4c6-534b-4900-a959-28fc0899c2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:18.009771 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ad4c6-534b-4900-a959-28fc0899c2b8 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:18.019646 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1468/2906] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:18.023300 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:18.032907 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c3733-8700-44f1-912c-b60028a51f10 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:18.037872 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:18.038717 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.038994 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.039267 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] PATH_INFO: `/v3/projects/df2c2d510f054ffe9d3bd2f2f73d7beb/users/b394d1eec43b463092afe8b0cc32ca85/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.053894 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df2c2d510f054ffe9d3bd2f2f73d7beb, user_id=b394d1eec43b463092afe8b0cc32ca85, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:18.056529 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:18.068553 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1439/2907] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:18 2022] PUT /identity/v3/projects/df2c2d510f054ffe9d3bd2f2f73d7beb/users/b394d1eec43b463092afe8b0cc32ca85/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:18.079642 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.079937 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.080253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.114693 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] MFA Rules not processed for user `b394d1eec43b463092afe8b0cc32ca85`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:18.115934 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:18.179465 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1469/2908] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:18.190643 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:18.192447 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:18.247934 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7b48a-5b56-4b79-95af-9cd32c29f94d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:18.251674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf326477-0302-4a0b-a00d-c84017740653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:18.252348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf326477-0302-4a0b-a00d-c84017740653 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.252638 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf326477-0302-4a0b-a00d-c84017740653 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.252924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf326477-0302-4a0b-a00d-c84017740653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.261161 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf326477-0302-4a0b-a00d-c84017740653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:18.262608 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf326477-0302-4a0b-a00d-c84017740653 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:18.272007 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1440/2909] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:18.291354 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:18.353598 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910774f-710d-4cab-aa6f-31aee992ade5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:18.358253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:18.359142 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.359428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.359706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] PATH_INFO: `/v3/users/b394d1eec43b463092afe8b0cc32ca85` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.360923 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b394d1eec43b463092afe8b0cc32ca85)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:18.362951 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:18.384796 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:18.411612 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b394d1eec43b463092afe8b0cc32ca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.418394 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b394d1eec43b463092afe8b0cc32ca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.419024 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b394d1eec43b463092afe8b0cc32ca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.430652 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b394d1eec43b463092afe8b0cc32ca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.437423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b394d1eec43b463092afe8b0cc32ca85'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.443565 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1470/2910] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:18 2022] DELETE /identity/v3/users/b394d1eec43b463092afe8b0cc32ca85 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:18.448230 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.448547 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.448806 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.454430 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:18.468461 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:18.469666 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:18.528067 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1441/2911] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:18.538943 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:18.540389 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:18.599530 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bb4a8-c8e4-4941-bb0a-8595959bf0b9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:18.603201 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4652795-656e-45b2-a2be-c1aea605a33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:18.603852 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4652795-656e-45b2-a2be-c1aea605a33c service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.604138 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4652795-656e-45b2-a2be-c1aea605a33c service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.604533 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4652795-656e-45b2-a2be-c1aea605a33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.615752 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4652795-656e-45b2-a2be-c1aea605a33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:18.617221 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4652795-656e-45b2-a2be-c1aea605a33c service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:18.626543 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1471/2912] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:18.887750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:18.889248 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:18.901976 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f5aeb-b9a6-4630-904b-92c81b501834 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:18.906538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:18.907408 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.907691 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.907968 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] PATH_INFO: `/v3/projects/df2c2d510f054ffe9d3bd2f2f73d7beb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:18.909229 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df2c2d510f054ffe9d3bd2f2f73d7beb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:18.911117 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:18.958231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df2c2d510f054ffe9d3bd2f2f73d7beb', 'request_id': 'req-0e5d369d-3c92-45e6-88cb-796ec42fc464'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.963750 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df2c2d510f054ffe9d3bd2f2f73d7beb', 'request_id': 'req-0e5d369d-3c92-45e6-88cb-796ec42fc464'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.970809 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df2c2d510f054ffe9d3bd2f2f73d7beb', 'request_id': 'req-0e5d369d-3c92-45e6-88cb-796ec42fc464'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.981711 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] The token cache is being invalidate because project df2c2d510f054ffe9d3bd2f2f73d7beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:18.981998 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.982609 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0e5d369d-3c92-45e6-88cb-796ec42fc464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:18.984122 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1442/2913] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:18 2022] DELETE /identity/v3/projects/df2c2d510f054ffe9d3bd2f2f73d7beb => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:18.989741 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:18.990032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:18.990310 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.007974 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:19.009187 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:19.068235 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1472/2914] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:19.075805 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.076090 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.076398 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.093949 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:19.095128 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:19.102813 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1443/2915] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:19.106733 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.116576 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29aa249-dfc9-406c-9633-cdb4e50b24fd None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.121048 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.121690 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.121973 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.122250 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] PATH_INFO: `/v3/domains` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.122963 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80171) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 19 13:11:19.123267 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.124884 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.137756 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1473/2916] 10.222.0.13 () {58 vars in 1152 bytes} [Tue Jul 19 13:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 19 13:11:19.142061 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.151337 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe0c96-c405-4b62-ab12-e119a5f19d7d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.155954 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.156629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.156917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.157197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.157891 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.159759 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.184778 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1444/2917] 10.222.0.13 () {60 vars in 1152 bytes} [Tue Jul 19 13:11:19 2022] POST /identity/v3/projects => generated 361 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 19 13:11:19.188794 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.199339 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc321-e59f-4d65-9f73-be6815f4214e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.227302 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:19.256403 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.257073 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.257357 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.257639 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.258326 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.260277 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.279746 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1474/2918] 10.222.0.13 () {60 vars in 1143 bytes} [Tue Jul 19 13:11:19 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 19 13:11:19.283876 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.293992 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60b494-a3d2-42a6-a498-354b92ec6958 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.298581 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.299748 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.300034 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.300349 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.301069 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.302677 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.312904 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1445/2919] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:19.317266 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.327064 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa752678-1fc4-45aa-86e1-599593792b2e admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.331671 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8588ca10-1c47-4aa6-81d9-b8799eba78f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.332348 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588ca10-1c47-4aa6-81d9-b8799eba78f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.332640 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588ca10-1c47-4aa6-81d9-b8799eba78f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.332925 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588ca10-1c47-4aa6-81d9-b8799eba78f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.333637 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588ca10-1c47-4aa6-81d9-b8799eba78f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.335226 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8588ca10-1c47-4aa6-81d9-b8799eba78f1 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.344972 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1475/2920] 10.222.0.13 () {58 vars in 1121 bytes} [Tue Jul 19 13:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 19 13:11:19.349348 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.359110 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c68c4-5c70-417e-8571-adca92419dae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.364195 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.365076 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.365365 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.365644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] PATH_INFO: `/v3/projects/5ad4a0941fe14a33b7767fcced9ada04/users/c2d14839145c492a96f08b4348e7e4ec/roles/13dc310ce9ce466088231180b4040e69` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.379967 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad4a0941fe14a33b7767fcced9ada04, user_id=c2d14839145c492a96f08b4348e7e4ec, role_id=13dc310ce9ce466088231180b4040e69)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.382640 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.395999 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1446/2921] 10.222.0.13 () {60 vars in 1482 bytes} [Tue Jul 19 13:11:19 2022] PUT /identity/v3/projects/5ad4a0941fe14a33b7767fcced9ada04/users/c2d14839145c492a96f08b4348e7e4ec/roles/13dc310ce9ce466088231180b4040e69 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:19.407393 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.407842 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.408136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.435402 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] MFA Rules not processed for user `c2d14839145c492a96f08b4348e7e4ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:19.436649 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:19.499500 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1476/2922] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:19.529177 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.530902 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:19.558525 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:19.591309 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b87526-68a4-428d-8e4d-055829890570 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.594970 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b80b12-d61d-45bc-b14c-218ea3546117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a58d153dd25943ef80eea95893118e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.595613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b80b12-d61d-45bc-b14c-218ea3546117 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.596421 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b80b12-d61d-45bc-b14c-218ea3546117 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.596705 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b80b12-d61d-45bc-b14c-218ea3546117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.604997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b80b12-d61d-45bc-b14c-218ea3546117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.606466 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b80b12-d61d-45bc-b14c-218ea3546117 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.616294 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1447/2923] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:19.631098 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.693670 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220ec6e-9ca3-41dc-ba6d-8f96b995a3b5 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.698161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.699011 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.699292 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.699567 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] PATH_INFO: `/v3/users/c2d14839145c492a96f08b4348e7e4ec` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.700764 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d14839145c492a96f08b4348e7e4ec)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.702719 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.743834 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:19.749582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d14839145c492a96f08b4348e7e4ec'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:19.757607 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d14839145c492a96f08b4348e7e4ec'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:19.758385 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d14839145c492a96f08b4348e7e4ec'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:19.770855 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d14839145c492a96f08b4348e7e4ec'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:19.777574 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d14839145c492a96f08b4348e7e4ec'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:19.784588 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1477/2924] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:11:19 2022] DELETE /identity/v3/users/c2d14839145c492a96f08b4348e7e4ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:19.789051 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.789345 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.789630 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.806539 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] MFA Rules not processed for user `f3447cd66131465b871daa6541b4276c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:11:19.807724 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:19.871590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1448/2925] 10.222.0.13 () {58 vars in 957 bytes} [Tue Jul 19 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:11:19.881448 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:19.882876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:19.940013 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af2165-b60f-4a22-a129-0736456011e9 admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:19.943692 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b269d-dcb7-4f83-939a-6d5ffbd37adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:19.944377 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b269d-dcb7-4f83-939a-6d5ffbd37adf service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:19.944668 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b269d-dcb7-4f83-939a-6d5ffbd37adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:19.944946 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b269d-dcb7-4f83-939a-6d5ffbd37adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:19.953882 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b269d-dcb7-4f83-939a-6d5ffbd37adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:19.955850 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b269d-dcb7-4f83-939a-6d5ffbd37adf service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:19.965246 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1478/2926] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:11:20.231997 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:11:20.233554 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:11:20.247108 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c535-2473-4efb-996a-43fac840c36a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:11:20.252488 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:11:20.253356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:11:20.253644 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:11:20.253924 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] PATH_INFO: `/v3/projects/5ad4a0941fe14a33b7767fcced9ada04` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:11:20.255121 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad4a0941fe14a33b7767fcced9ada04)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:11:20.257071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:11:20.304485 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad4a0941fe14a33b7767fcced9ada04', 'request_id': 'req-d4afd0d8-4f7b-4ae6-916a-197528b114cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:20.310018 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad4a0941fe14a33b7767fcced9ada04', 'request_id': 'req-d4afd0d8-4f7b-4ae6-916a-197528b114cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:20.318402 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad4a0941fe14a33b7767fcced9ada04', 'request_id': 'req-d4afd0d8-4f7b-4ae6-916a-197528b114cd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:20.329396 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] The token cache is being invalidate because project 5ad4a0941fe14a33b7767fcced9ada04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:11:20.329681 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:20.330852 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4afd0d8-4f7b-4ae6-916a-197528b114cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:11:20.332441 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1449/2927] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:11:20 2022] DELETE /identity/v3/projects/5ad4a0941fe14a33b7767fcced9ada04 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:11:29.961177 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:11:30.326361 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:12:02.780443 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ec2c-8be8-4789-ac3a-673f6559dbe0 None None] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:02.780994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ec2c-8be8-4789-ac3a-673f6559dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:02.780994 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5ec2c-8be8-4789-ac3a-673f6559dbe0 None None] PATH_INFO: `/` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:02.782969 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1479/2928] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 13:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 13:12:02.789382 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348075d-bc21-49c9-bb52-9c1a8b4ba010 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:02.789828 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348075d-bc21-49c9-bb52-9c1a8b4ba010 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:02.790113 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348075d-bc21-49c9-bb52-9c1a8b4ba010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:02.811002 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8348075d-bc21-49c9-bb52-9c1a8b4ba010 None None] MFA Rules not processed for user `fd55fa1fab59477f8d747cdf3fdeb7f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:12:02.812328 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8348075d-bc21-49c9-bb52-9c1a8b4ba010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:12:02.866220 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1450/2929] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 13:12:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:12:02.881253 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5ec2c-8be8-4789-ac3a-673f6559dbe0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:12:02.883587 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-dfd5ec2c-8be8-4789-ac3a-673f6559dbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:12:02.898074 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5ec2c-8be8-4789-ac3a-673f6559dbe0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:12:02.901773 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cb64a-44db-4911-9cd1-9f770fdf6d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:12:02.902431 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cb64a-44db-4911-9cd1-9f770fdf6d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:02.902817 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cb64a-44db-4911-9cd1-9f770fdf6d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:02.902993 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cb64a-44db-4911-9cd1-9f770fdf6d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:02.913147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cb64a-44db-4911-9cd1-9f770fdf6d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:12:02.913147 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cb64a-44db-4911-9cd1-9f770fdf6d57 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:12:02.920254 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1480/2930] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:12:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:12:03.001197 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751973-bfdf-4e21-b76e-15035769b6dd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:03.001491 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751973-bfdf-4e21-b76e-15035769b6dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:03.001771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751973-bfdf-4e21-b76e-15035769b6dd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:03.003200 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1451/2931] 10.222.0.13 () {54 vars in 921 bytes} [Tue Jul 19 13:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 19 13:12:03.008337 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] REQUEST_METHOD: `POST` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:03.008634 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:03.008919 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:03.027120 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.auth.core [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] MFA Rules not processed for user `99d07f11438d4e069eab093a8b99eeab`. Rule list: `[]` (Enabled: `True`). {{(pid=80171) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 19 13:12:03.028349 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:12:03.092942 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1481/2932] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Jul 19 13:12:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 19 13:12:03.103590 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e751973-bfdf-4e21-b76e-15035769b6dd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:12:03.114917 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e751973-bfdf-4e21-b76e-15035769b6dd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:12:03.118450 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:12:03.119073 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:03.119356 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:03.119629 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:03.129288 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:12:03.130700 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:12:03.141049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1452/2933] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:12:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:12:13.083695 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=37 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:12:13.136538 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=29 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:12:44.392383 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:12:44.412275 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d81d94-1389-47fe-a373-e9025eaa89d0 None None] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:12:44.415783 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:12:44.416458 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:12:44.416748 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:12:44.417026 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:12:44.418775 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:12:44.428562 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:12:44.429988 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:12:44.438088 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1482/2934] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:12:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:12:54.434706 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=10 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:13:29.036344 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:29.037988 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:29.060863 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d0122-dd34-4724-8e6b-cd23648160b9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:29.064763 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:29.065780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:29.065780 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:29.065993 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:29.068436 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:29.093045 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:29.094511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:29.120771 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1453/2935] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:13:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:13:30.304032 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:30.312770 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bd1c5-ec21-42d8-8ce6-634644ca0ede service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:30.316427 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd55fa1fab59477f8d747cdf3fdeb7f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:30.317061 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:30.317347 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:30.317627 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:30.319371 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:30.332535 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:30.334449 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:30.345425 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1483/2936] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:13:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:13:35.837261 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:35.838699 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:35.930074 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020e950-e2f3-497e-9982-2e13d796d08c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:35.935180 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:35.935908 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:35.936306 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:35.936660 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:35.938580 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:35.952687 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:35.954282 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:35.965360 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1454/2937] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:13:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 19 13:13:38.977204 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:38.988928 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712e776-ccea-415b-8652-16a59941fae7 service neutron] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:38.994452 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:38.995089 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] REQUEST_METHOD: `GET` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:38.995370 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:38.995644 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:38.997384 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:39.008967 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:39.010501 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:39.020879 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1484/2938] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:13:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:13:39.044071 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:39.045645 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:39.111549 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddecf6f-46b0-43f8-ab61-1cc0b756b759 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:39.116049 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:39.116946 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:39.117231 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:39.117511 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] PATH_INFO: `/v3/users/97565560b942439c9c4c09bf3097b8c8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:39.118674 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97565560b942439c9c4c09bf3097b8c8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:39.120613 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:39.166048 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97565560b942439c9c4c09bf3097b8c8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.166672 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97565560b942439c9c4c09bf3097b8c8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.173651 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97565560b942439c9c4c09bf3097b8c8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.175109 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:13:39.183475 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97565560b942439c9c4c09bf3097b8c8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.195599 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97565560b942439c9c4c09bf3097b8c8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.201743 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1455/2939] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:13:39 2022] DELETE /identity/v3/users/97565560b942439c9c4c09bf3097b8c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:13:39.368619 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:39.381136 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610e041-aeeb-4867-88a2-57f85eb52192 service placement] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:39.438439 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:39.439298 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:39.439582 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:39.439858 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] PATH_INFO: `/v3/projects/3de65ab2754b49d69c41eb4d2870648e` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:39.441058 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de65ab2754b49d69c41eb4d2870648e)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:39.443481 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:39.489554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de65ab2754b49d69c41eb4d2870648e', 'request_id': 'req-f6c81808-368b-4415-8bf8-8f86f44aa34b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.496688 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de65ab2754b49d69c41eb4d2870648e', 'request_id': 'req-f6c81808-368b-4415-8bf8-8f86f44aa34b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.502899 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de65ab2754b49d69c41eb4d2870648e', 'request_id': 'req-f6c81808-368b-4415-8bf8-8f86f44aa34b'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.513554 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] The token cache is being invalidate because project 3de65ab2754b49d69c41eb4d2870648e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:13:39.513841 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.514466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.516063 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1485/2940] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:13:39 2022] DELETE /identity/v3/projects/3de65ab2754b49d69c41eb4d2870648e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 19 13:13:39.521127 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:39.522657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:39.537146 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21078810-25ce-41ed-847b-a7baf31aa22f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:39.593801 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:39.594665 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:39.594953 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:39.595235 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] PATH_INFO: `/v3/users/05886636d6ab456db9297e082dbf0920` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:39.596449 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05886636d6ab456db9297e082dbf0920)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:39.598367 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:39.649838 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05886636d6ab456db9297e082dbf0920'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.650498 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05886636d6ab456db9297e082dbf0920'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.661420 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05886636d6ab456db9297e082dbf0920'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.669238 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05886636d6ab456db9297e082dbf0920'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.680829 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05886636d6ab456db9297e082dbf0920'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.686657 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1456/2941] 10.222.0.13 () {58 vars in 1223 bytes} [Tue Jul 19 13:13:39 2022] DELETE /identity/v3/users/05886636d6ab456db9297e082dbf0920 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:13:39.859876 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Authenticating user token {{(pid=80171) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:13:39.861345 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.fernet_utils [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80171) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:13:39.877211 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c81808-368b-4415-8bf8-8f86f44aa34b admin admin] Validating token access rules against request {{(pid=80171) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:13:39.936306 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3447cd66131465b871daa6541b4276c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45290f5162bc4ec4b133291838122f88', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80171) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:13:39.937161 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:13:39.937428 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:13:39.937699 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] PATH_INFO: `/v3/projects/545016fd05654847998ed6f7ba9339fa` {{(pid=80171) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:13:39.938862 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=545016fd05654847998ed6f7ba9339fa)` {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:13:39.940798 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] RBAC: Authorization granted {{(pid=80171) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:13:39.947367 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:13:39.987753 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545016fd05654847998ed6f7ba9339fa', 'request_id': 'req-065af69b-6f70-4c74-a19b-23953ef2d3c5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:39.996790 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545016fd05654847998ed6f7ba9339fa', 'request_id': 'req-065af69b-6f70-4c74-a19b-23953ef2d3c5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:40.005623 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545016fd05654847998ed6f7ba9339fa', 'request_id': 'req-065af69b-6f70-4c74-a19b-23953ef2d3c5'} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:40.019466 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] The token cache is being invalidate because project 545016fd05654847998ed6f7ba9339fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80171) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 19 13:13:40.019749 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:40.020423 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG keystone.notifications [None req-065af69b-6f70-4c74-a19b-23953ef2d3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80171) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 19 13:13:40.021957 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: [pid: 80171|app: 0|req: 1486/2942] 10.222.0.13 () {58 vars in 1232 bytes} [Tue Jul 19 13:13:39 2022] DELETE /identity/v3/projects/545016fd05654847998ed6f7ba9339fa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 19 13:13:49.683227 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:13:50.016133 dv-cinder-820994-4-iscsi devstack@keystone.service[80171]: DEBUG dbcounter [-] [80171] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80171) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 19 13:14:48.643848 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 19 13:14:48.645960 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:14:48.717246 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7441d-fdb0-40f8-9d4e-632498990e12 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 19 13:14:48.775721 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06d3500f0bda4ec58b2d561f13eb42d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b1a666f3a1a144858853d303838a22b9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 19 13:14:48.776401 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 19 13:14:48.776688 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 19 13:14:48.776963 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 19 13:14:48.778719 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 19 13:14:48.789634 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 19 13:14:48.791189 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e1b07b-d355-4b51-a6fc-ce7e2a7e22eb service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 19 13:14:48.801322 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1457/2943] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Jul 19 13:14:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 19 13:14:58.796612 dv-cinder-820994-4-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}